Package io.deephaven.kafka.ingest

Interface ConsumerRecordToStreamPublisherAdapter

All Superinterfaces:
StreamPublisher
All Known Implementing Classes:
KafkaStreamPublisher
public interface ConsumerRecordToStreamPublisherAdapter extends StreamPublisher
Converter from a stream of Kafka records to a Deephaven StreamPublisher.

  • Method Summary

    Modifier and Type
    Method
    Description
    long
    consumeRecords(long receiveTime, @NotNull List<? extends org.apache.kafka.clients.consumer.ConsumerRecord<?,?>> records)
    Consume a List of Kafka records, producing zero or more rows in the output.
    void
    propagateFailure(@NotNull Throwable cause)
    Propagate a failure from the Kafka consumer to this StreamPublisher.

    Methods inherited from interface io.deephaven.stream.StreamPublisher

    flush, register, shutdown

  • Method Details

    • propagateFailure

      void propagateFailure(@NotNull @NotNull Throwable cause)
      Propagate a failure from the Kafka consumer to this StreamPublisher.
      Parameters:
      cause - The failure to propagate

    • consumeRecords

      long consumeRecords(long receiveTime, @NotNull @NotNull List<? extends org.apache.kafka.clients.consumer.ConsumerRecord<?,?>> records) throws IOException
      Consume a List of Kafka records, producing zero or more rows in the output.
      Parameters:
      receiveTime - the time, in nanoseconds since the epoch, the records were received in this process
      records - the records received from KafkaConsumer.poll(Duration).
      Returns:
      the number of bytes processed
      Throws:
      IOException - if there was an error writing to the output table