Package io.deephaven.qst.table

Class TableBase

java.lang.Object
io.deephaven.qst.table.TableBase
All Implemented Interfaces:
TableOperations<TableSpec,TableSpec>, TableOperationsDefaults<TableSpec,TableSpec>, TableSchema, TableSpec
Direct Known Subclasses:
ByTableBase, DropColumnsTable, EmptyTable, HeadTable, InputTableBase, JoinBase, LazyUpdateTable, MergeTable, MultiJoinTable, NewTable, RangeJoinTable, ReverseTable, SelectDistinctTable, SelectTable, SnapshotTable, SnapshotWhenTable, SortTable, TailTable, TicketTable, TimeTable, UngroupTable, UpdateTable, UpdateViewTable, ViewTable, WhereInTable, WhereTable
public abstract class TableBase extends Object implements TableSpec

  • Constructor Details

    • TableBase

      public TableBase()

  • Method Details

    • logic

      public final TableCreationLogic logic()
      Specified by:
      logic in interface TableSpec

    • head

      public final TableSpec head(long size)
      Specified by:
      head in interface TableOperations<TableSpec,TableSpec>

    • tail

      public final TableSpec tail(long size)
      Specified by:
      tail in interface TableOperations<TableSpec,TableSpec>

    • reverse

      public final TableSpec reverse()
      Specified by:
      reverse in interface TableOperations<TableSpec,TableSpec>

    • snapshot

      public final TableSpec snapshot()
      Description copied from interface: TableOperations
      Creates a table with a single static snapshot of this.
      Specified by:
      snapshot in interface TableOperations<TableSpec,TableSpec>
      Returns:
      the snapshot

    • snapshotWhen

      public final TableSpec snapshotWhen(TableSpec trigger, SnapshotWhenOptions.Flag... features)
      Description copied from interface: TableOperations
      Creates a table that captures a snapshot of this whenever trigger updates.

      Equivalent to snapshotWhen(trigger, SnapshotWhenControl.of(features)).

      Specified by:
      snapshotWhen in interface TableOperations<TableSpec,TableSpec>
      Parameters:
      trigger - the trigger table
      features - the snapshot features
      Returns:
      the snapshotting table
      See Also:

    • snapshotWhen

      public final TableSpec snapshotWhen(TableSpec trigger, Collection<SnapshotWhenOptions.Flag> features, String... stampColumns)
      Description copied from interface: TableOperations
      Creates a table that captures a snapshot of this whenever trigger updates.

      Equivalent to snapshotWhen(trigger, SnapshotWhenControl.of(features, stampColumns)).

      See SnapshotWhenOptions for details on the stampColumns.

      Specified by:
      snapshotWhen in interface TableOperations<TableSpec,TableSpec>
      Parameters:
      trigger - the trigger table
      features - the snapshot features
      stampColumns - the stamp columns
      Returns:
      the snapshotting table
      See Also:

    • snapshotWhen

      public final TableSpec snapshotWhen(TableSpec trigger, SnapshotWhenOptions options)
      Description copied from interface: TableOperations
      Creates a table that captures a snapshot of this whenever trigger updates.
      Specified by:
      snapshotWhen in interface TableOperations<TableSpec,TableSpec>
      Parameters:
      trigger - the trigger table
      options - the snapshot options
      Returns:
      the snapshotting table
      See Also:

    • sort

      public final TableSpec sort(Collection<SortColumn> columnsToSortBy)
      Specified by:
      sort in interface TableOperations<TableSpec,TableSpec>

    • where

      public final TableSpec where(Filter filter)
      Specified by:
      where in interface TableOperations<TableSpec,TableSpec>

    • whereIn

      public final TableSpec whereIn(TableSpec rightTable, Collection<? extends JoinMatch> columnsToMatch)
      Description copied from interface: TableOperations
      Filters this table based on the set of values in the rightTable.

      Note that when the rightTable ticks, all of the rows in this table are going to be re-evaluated, thus the intention is that the rightTable is fairly slow moving compared with this table.

      Specified by:
      whereIn in interface TableOperations<TableSpec,TableSpec>
      Parameters:
      rightTable - the filtering table.
      columnsToMatch - the columns to match between the two tables
      Returns:
      a new table filtered on right table

    • whereNotIn

      public final TableSpec whereNotIn(TableSpec rightTable, Collection<? extends JoinMatch> columnsToMatch)
      Description copied from interface: TableOperations
      Filters this table based on the set of values not in the rightTable.

      Note that when the rightTable ticks, all of the rows in this table are going to be re-evaluated, thus the intention is that the rightTable is fairly slow moving compared with this table.

      Specified by:
      whereNotIn in interface TableOperations<TableSpec,TableSpec>
      Parameters:
      rightTable - the filtering table.
      columnsToMatch - the columns to match between the two tables
      Returns:
      a new table filtered on right table

    • naturalJoin

      public final TableSpec naturalJoin(TableSpec rightTable, Collection<? extends JoinMatch> columnsToMatch, Collection<? extends JoinAddition> columnsToAdd)
      Description copied from interface: TableOperations
      Perform an exact-join with the rightTable.

      Requires zero or one match from the rightTable.

      Specified by:
      naturalJoin in interface TableOperations<TableSpec,TableSpec>
      Parameters:
      rightTable - The right side table on the join.
      columnsToMatch - The match pair conditions.
      columnsToAdd - The columns from the right side that need to be added to the left side as a result of the match.
      Returns:
      the natural-joined table

    • exactJoin

      public final TableSpec exactJoin(TableSpec rightTable, Collection<? extends JoinMatch> columnsToMatch, Collection<? extends JoinAddition> columnsToAdd)
      Description copied from interface: TableOperations
      Perform an exact-join with the rightTable.

      Similar to TableOperations.naturalJoin(Object, Collection, Collection), but requires that exactly one match from the rightTable.

      Specified by:
      exactJoin in interface TableOperations<TableSpec,TableSpec>
      Parameters:
      rightTable - The right side table on the join.
      columnsToMatch - The match pair conditions.
      columnsToAdd - The columns from the right side that need to be added to the left side as a result of the match.
      Returns:
      the exact-joined table

    • join

      public final TableSpec join(TableSpec rightTable, Collection<? extends JoinMatch> columnsToMatch, Collection<? extends JoinAddition> columnsToAdd)
      Description copied from interface: TableOperations
      Perform a cross join with the rightTable.

      Delegates to TableOperations.join(Object, Collection, Collection, int).

      Specified by:
      join in interface TableOperations<TableSpec,TableSpec>
      Parameters:
      rightTable - The right side table on the join.
      columnsToMatch - The match pair conditions.
      columnsToAdd - The columns from the right side that need to be added to the left side as a result of the match.
      Returns:
      a new table joined according to the specification in columnsToMatch and columnsToAdd

    • join

      public final TableSpec join(TableSpec rightTable, Collection<? extends JoinMatch> columnsToMatch, Collection<? extends JoinAddition> columnsToAdd, int reserveBits)
      Description copied from interface: TableOperations
      Perform a cross join with the rightTable.

      Returns a table that is the cartesian product of left rows X right rows, with one column for each of this table's columns, and one column corresponding to each of the rightTable's columns that are included in the columnsToAdd argument. The rows are ordered first by the this table then by the rightTable. If columnsToMatch is non-empty then the product is filtered by the supplied match conditions.

      To efficiently produce updates, the bits that represent a key for a given row are split into two. Unless specified, join reserves 16 bits to represent a right row. When there are too few bits to represent all of the right rows for a given aggregation group the table will shift a bit from the left side to the right side. The default of 16 bits was carefully chosen because it results in an efficient implementation to process live updates.

      An io.deephaven.engine.table.impl.util.OutOfKeySpaceException is thrown when the total number of bits needed to express the result table exceeds that needed to represent Long.MAX_VALUE. There are a few work arounds:

      - If the left table is sparse, consider flattening the left table.

      - If there are no key-columns and the right table is sparse, consider flattening the right table.

      - If the maximum size of a right table's group is small, you can reserve fewer bits by setting reserveBits on initialization.

      Note: If you know that a given group has at most one right-row then you should prefer using TableOperations.naturalJoin(Object, Collection, Collection).

      Specified by:
      join in interface TableOperations<TableSpec,TableSpec>
      Parameters:
      rightTable - The right side table on the join.
      columnsToMatch - The match pair conditions.
      columnsToAdd - The columns from the right side that need to be added to the left side as a result of the match.
      reserveBits - The number of bits to reserve for rightTable groups.
      Returns:
      a new table joined according to the specification in columnsToMatch and columnsToAdd

    • asOfJoin

      public final TableSpec asOfJoin(TableSpec rightTable, Collection<? extends JoinMatch> exactMatches, AsOfJoinMatch asOfMatch, Collection<? extends JoinAddition> columnsToAdd)
      Specified by:
      asOfJoin in interface TableOperations<TableSpec,TableSpec>

    • rangeJoin

      public final TableSpec rangeJoin(TableSpec rightTable, Collection<? extends JoinMatch> exactMatches, RangeJoinMatch rangeMatch, Collection<? extends Aggregation> aggregations)
      Description copied from interface: TableOperations
      Perform a range join with rightTable. For each row in this Table, this operation joins aggregations over a range of responsive rows from rightTable according to zero-or-more exact join matches and one range join match.

      Matching Rules

      The exact join matches identify possibly-responsive rows according to exactly matching values between the left exact match columns and the right exact match columns, similar to other join operations. The range join match bounds the beginning of the responsive range for a given output row by the relationship between a left start column and a right range column, governed by the rangeStartRule, and bounds the end of the responsive range for a given output row by the relationship between a left end column and the same right range column, governed by the rangeEndRule.

      Right Table Row-Inclusion Criteria and Relative Ordering Requirements

      Rows from rightTable with null or NaN values for the right range column are discarded; that is, they are never included in the responsive range for any output row. Within a group sharing the same values for the right exact match columns, rightTable must be relatively ordered (as if sorted) according to the right range column for all rows that are not discarded.

      Special Cases

      In order to produce aggregated output, it is required that the two relative match expressions define a range of values to determine the responsive rows to aggregate. There are a few noteworthy special cases of ranges.

      empty range
      An empty range occurs for any left row with no responsive right rows. That is, no non-null, non-NaN right rows were found using the exact join matches, or none were in range according to the range join match.
      single-value ranges
      A single-value range is a range where the left row’s values for the left start column and left end column are equal and both relative matches are inclusive (<= and >=, respectively). For a single-value range, only rows within the bucket where the right range column matches the single value are included in the output aggregations.
      invalid ranges
      An invalid range occurs in two scenarios: First, when the range is inverted, i.e. when the value of the left start column is greater than the value of the left end column. Second, when either relative-match is exclusive (< or >) and the value in the left start column is equal to the value in the left end column (because value < value == false). Specifying "allow preceding" or "allow following" for either rule will not constitute an exception to either of these defined scenarios. For invalid ranges, the result row will be null for all aggregation output columns.
      undefined ranges
      An undefined range occurs when either the left start column or the left end column is NaN. For rows with an undefined range, the corresponding output values will be null (as with invalid ranges).
      Unbounded Ranges
      A partially or fully unbounded range occurs when either the left start column or the left end column is null. If the left start column value is null the range is unbounded at the beginning, and all matched right rows will be included if they respect the match rule for the left end column. If the left end column value is null the range is unbounded at the end, and all matched right rows will be included if they respect the match rule for the left start column. If both the left start column and left end column values are null the range is unbounded in both directions, and all matched right rows will be included.
      Specified by:
      rangeJoin in interface TableOperations<TableSpec,TableSpec>
      Parameters:
      rightTable - The Table to join with
      exactMatches - Possibly-empty collection of join matches that dictate exact-match criteria. That is, rows from rightTable that might be responsive to rows from this Table will have identical values for the column pairs expressed by these matches.
      rangeMatch - Specifies the range match criteria for determining the responsive rows from rightTable for each row from this Table, within the buckets created by matching on the exactMatches
      aggregations - The aggregations to perform over the responsive ranges from rightTable for each row from this Table
      Returns:
      The result Table

    • view

      public final TableSpec view(Collection<? extends Selectable> columns)
      Specified by:
      view in interface TableOperations<TableSpec,TableSpec>

    • updateView

      public final TableSpec updateView(Collection<? extends Selectable> columns)
      Specified by:
      updateView in interface TableOperations<TableSpec,TableSpec>

    • update

      public final TableSpec update(Collection<? extends Selectable> columns)
      Specified by:
      update in interface TableOperations<TableSpec,TableSpec>

    • lazyUpdate

      public final TableSpec lazyUpdate(Collection<? extends Selectable> columns)
      Description copied from interface: TableOperations
      Compute column formulas on demand.

      Lazy update defers computation until required for a set of values, and caches the results for a set of input values. This uses less RAM than an update statement when you have a smaller set of unique values. Less computation than an updateView is needed, because the results are saved in a cache.

      If you have many unique values, you should instead use an update statement, which will have more memory efficient structures. Values are never removed from the lazyUpdate cache, so it should be used judiciously on a ticking table.

      Specified by:
      lazyUpdate in interface TableOperations<TableSpec,TableSpec>
      Parameters:
      columns - the columns to add
      Returns:
      a new Table with the columns added; to be computed on demand

    • select

      public final TableSpec select()
      Specified by:
      select in interface TableOperations<TableSpec,TableSpec>
      Specified by:
      select in interface TableOperationsDefaults<TableSpec,TableSpec>

    • select

      public final TableSpec select(Collection<? extends Selectable> columns)
      Specified by:
      select in interface TableOperations<TableSpec,TableSpec>

    • aggAllBy

      public final TableSpec aggAllBy(AggSpec spec, ColumnName... groupByColumns)
      Specified by:
      aggAllBy in interface TableOperations<TableSpec,TableSpec>

    • aggBy

      public TableSpec aggBy(Collection<? extends Aggregation> aggregations, boolean preserveEmpty, TableSpec initialGroups, Collection<? extends ColumnName> groupByColumns)
      Description copied from interface: TableOperations
      Produce an aggregated result by grouping this according to the groupByColumns and applying aggregations to each resulting group of rows. The result table will have one row per group, ordered by the encounter order within this, thereby ensuring that the row key for a given group never changes.
      Specified by:
      aggBy in interface TableOperations<TableSpec,TableSpec>
      Parameters:
      aggregations - The aggregations to apply
      preserveEmpty - Whether to keep result rows for groups that are initially empty or become empty as a result of updates. Each aggregation operator defines its own value for empty groups.
      initialGroups - A table whose distinct combinations of values for the groupByColumns should be used to create an initial set of aggregation groups. All other columns are ignored. This is useful in combination with preserveEmpty == true to ensure that particular groups appear in the result table, or with preserveEmpty == false to control the encounter order for a collection of groups and thus their relative order in the result. Changes to initialGroups are not expected or handled; if initialGroups is a refreshing table, only its contents at instantiation time will be used. If initialGroups == null, the result will be the same as if a table with no rows was supplied.
      groupByColumns - The columns to group by
      Returns:
      A new table aggregating the rows of this

    • updateBy

      public final TableSpec updateBy(UpdateByControl control, Collection<? extends UpdateByOperation> operations, Collection<? extends ColumnName> byColumns)
      Description copied from interface: TableOperations
      Creates a table with additional columns calculated from window-based aggregations of columns in its parent. The aggregations are defined by the operations, which support incremental aggregation over the corresponding rows in the parent table. The aggregations will apply position or time-based windowing and compute the results for the row group (as determined by the byColumns).
      Specified by:
      updateBy in interface TableOperations<TableSpec,TableSpec>
      Parameters:
      control - the control to use when updating the table.
      operations - the operations to apply to the table.
      byColumns - the columns to group by before applying.
      Returns:
      a table with the same rowSet, with the specified operations applied to each group defined by the byColumns

    • selectDistinct

      public final TableSpec selectDistinct()
      Specified by:
      selectDistinct in interface TableOperations<TableSpec,TableSpec>

    • selectDistinct

      public final TableSpec selectDistinct(Collection<? extends Selectable> columns)
      Specified by:
      selectDistinct in interface TableOperations<TableSpec,TableSpec>

    • ungroup

      public final TableSpec ungroup(boolean nullFill, Collection<? extends ColumnName> columnsToUngroup)
      Description copied from interface: TableOperations
      Ungroups a table by expanding columns of arrays or vectors into columns of singular values, creating one row in the output table for each value in the columns to be ungrouped. Columns that are not ungrouped have their values duplicated in each output row corresponding to a given input row.
      Specified by:
      ungroup in interface TableOperations<TableSpec,TableSpec>
      Parameters:
      nullFill - indicates if the ungrouped table should allow disparate sized arrays filling shorter columns with null values. If set to false, then all arrays should be the same length.
      columnsToUngroup - the columns to ungroup
      Returns:
      the ungrouped table

    • dropColumns

      public final TableSpec dropColumns(String... columnNames)
      Description copied from interface: TableOperations
      Creates a new table without the columnNames from this.
      Specified by:
      dropColumns in interface TableOperations<TableSpec,TableSpec>
      Parameters:
      columnNames - the columns to drop
      Returns:
      the table

    • walk

      public final <T> T walk(TableSchema.Visitor<T> visitor)
      Specified by:
      walk in interface TableSchema

    • toString

      public final String toString()
      Overrides:
      toString in class Object