consumeToPartitionedTable
The consumeToPartitionedTable method reads a Kafka stream into an in-memory partitioned table.
Syntax
Parameters
| Parameter | Type | Description |
|---|---|---|
| kafkaProperties | Properties | Configuration for the associated Kafka consumer and the resulting table. Once the table-specific properties are stripped, the remaining one is used to call the constructor of |
| topic | str | The Kafka topic name. |
| partitionFilter | IntPredicate | A predicate returning true for the partitions to consume. The convenience constant ALL_PARTITIONS is defined to facilitate requesting all partitions. |
| partitionToInitialOffset | IntToLongFunction | A function specifying the desired initial offset for each partition consumed. |
| keySpec | KeyOrValueSpec | Conversion specification for Kafka record keys. |
| valueSpec | KeyOrValueSpec | Conversion specification for Kafka record values. |
| tableType | TableType | A
|
Returns
An in-memory partitioned table.
Examples
In the following example, consumeToPartitionedTable is used to read the Kafka topic orders into a Deephaven table. The example ignores the Kafka key with KafkaTools.Consume.IGNORE and reads the value as JSON with KafkaTools.Consume.jsonSpec, which maps the JSON fields jsymbol, jside, jprice, and jqty to the Symbol, Side, Price, and Qty columns.
- The host and port for the Kafka server to use to bootstrap are specified by
kafkaProps.- The value
redpanda:9092corresponds to the current setup for development testing with Docker images (which uses an instance of redpanda).
- The value
- The topic name is
orders.