Package io.deephaven.engine.table.impl.util

Class SyncTableFilter

java.lang.Object
io.deephaven.engine.table.impl.util.SyncTableFilter
public class SyncTableFilter extends Object
Return the rows with the highest commonly available ID from multiple tables.

The Deephaven system does not provide cross table (or partition) transactions. However, you may have a producer that generates more than one table, which you would like to then use in a coordinated fashion. The Deephaven system preserves order within one partition, but the relative order of tables can not be guaranteed. The tailers, DIS, and other infrastructure process each partition independently to provide maximum throughput; and thus a row that logged later may appear in your query before another row that was logged earlier within a different partition.

If you tag each row with a long column that can be used to correlate the two tables, the SyncTableFilter can release only rows with matching values in all the input tables. For example, if you have input tables "a" "b" and "c", with a key of "USym", if "a" has rows for SPY with IDs of 1, 2, and 3, but "b" and "c" only have rows for "2", the filter will pass through the rows with ID 2. When both "b" and "c" have SPY rows for 3, then the rows for ID 2 are removed and the rows for ID 3 are added to the result tables.

The SyncTableFilter is configured through the Builder inner class. Tables are added to the builder, providing a name for each table. The return value is a map of the results with each filtered input table accessible according to the name provided to the builder.

For each key, only pass through the rows that have the minimum value of an ID across all tables. The IDs must be monotonically increasing for each key. Your underlying tables must make use of transactions such that all rows for a given ID appear in a input table at once. Please consult your Deephaven representative before deploying a query that includes this filter to ensure that the assumptions are not violated.