Package io.deephaven.engine.table

Interface Table

All Superinterfaces:
AttributeMap<Table>, DynamicNode, GridAttributes<Table>, LivenessManager, LivenessNode, LivenessReferent, LogOutputAppendable, LongSizedDataStructure, NotificationQueue.Dependency, SystemicObject<Table>, TableOperations<Table,​Table>
All Known Subinterfaces:
LazySnapshotTable, TableAdapter, TableDefaults
All Known Implementing Classes:
AppendOnlyArrayBackedMutableTable, BarrageRedirectedTable, BarrageStreamTable, BarrageTable, BaseTable, DeferredViewTable, InitialSnapshotTable, InMemoryTable, KeyedArrayBackedMutableTable, PartitionAwareSourceTable, QueryReplayGroupedTable, QueryTable, QueryTable.FilteredTable, RedefinableTable, ReplayGroupedFullTable, ReplayLastByGroupedTable, ReplayTable, SimpleSourceTable, SourceTable, TimeTable, UncoalescedTable, UpdatableTable, UpdateSourceQueryTable
public interface Table
extends LongSizedDataStructure, LivenessNode, NotificationQueue.Dependency, DynamicNode, SystemicObject<Table>, TableOperations<Table,​Table>, AttributeMap<Table>, GridAttributes<Table>
A Deephaven table.

  • Field Details

    • INPUT_TABLE_ATTRIBUTE

      static final String INPUT_TABLE_ATTRIBUTE
      See Also:
      Constant Field Values

    • KEY_COLUMNS_ATTRIBUTE

      static final String KEY_COLUMNS_ATTRIBUTE
      See Also:
      Constant Field Values

    • UNIQUE_KEYS_ATTRIBUTE

      static final String UNIQUE_KEYS_ATTRIBUTE
      See Also:
      Constant Field Values

    • FILTERABLE_COLUMNS_ATTRIBUTE

      static final String FILTERABLE_COLUMNS_ATTRIBUTE
      See Also:
      Constant Field Values

    • TOTALS_TABLE_ATTRIBUTE

      static final String TOTALS_TABLE_ATTRIBUTE
      See Also:
      Constant Field Values

    • ADD_ONLY_TABLE_ATTRIBUTE

      static final String ADD_ONLY_TABLE_ATTRIBUTE
      See Also:
      Constant Field Values

    • STREAM_TABLE_ATTRIBUTE

      static final String STREAM_TABLE_ATTRIBUTE

      If this attribute is present with value true, this Table is a "stream table".

      A stream table is a sequence of additions that represent rows newly received from a stream; on the cycle after the stream table is refreshed the rows are removed. Note that this means any particular row of data (not to be confused with a row key) never exists for more than one cycle.

      Most operations are supported as normal on stream tables, but aggregation operations are treated specially, producing aggregate results that are valid over the entire observed stream from the time the operation is initiated. These semantics necessitate a few exclusions, i.e. unsupported operations that need to keep track of all rows:

      1. TableOperations.groupBy() is unsupported
      2. partitionBy(boolean, java.lang.String...) is unsupported
      3. partitionedAggBy is unsupported
      4. TableOperations.aggBy(io.deephaven.api.agg.Aggregation) is unsupported if either of group or partition are used
      5. rollup() is unsupported if includeConstituents == true
      6. tree() is unsupported

      To disable these semantics, a dropStream method is offered.

      See Also:
      Constant Field Values

    • SORTED_COLUMNS_ATTRIBUTE

      static final String SORTED_COLUMNS_ATTRIBUTE
      The query engine may set or read this attribute to determine if a table is sorted by a particular column.
      See Also:
      Constant Field Values

    • SYSTEMIC_TABLE_ATTRIBUTE

      static final String SYSTEMIC_TABLE_ATTRIBUTE
      See Also:
      Constant Field Values

    • AGGREGATION_ROW_LOOKUP_ATTRIBUTE

      static final String AGGREGATION_ROW_LOOKUP_ATTRIBUTE
      Attribute on aggregation results used for hierarchical table construction. Specification is left to the implementation.
      See Also:
      Constant Field Values

    • SORT_REVERSE_LOOKUP_ATTRIBUTE

      static final String SORT_REVERSE_LOOKUP_ATTRIBUTE
      Attribute on sort results used for hierarchical table construction. Specification is left to the implementation.
      See Also:
      Constant Field Values

    • SNAPSHOT_VIEWPORT_TYPE

      static final String SNAPSHOT_VIEWPORT_TYPE
      See Also:
      Constant Field Values

    • MERGED_TABLE_ATTRIBUTE

      static final String MERGED_TABLE_ATTRIBUTE
      This attribute is used internally by TableTools.merge to detect successive merges. Its presence indicates that it is safe to decompose the table into its multiple constituent parts.
      See Also:
      Constant Field Values

    • INITIALLY_EMPTY_COALESCED_SOURCE_TABLE_ATTRIBUTE

      static final String INITIALLY_EMPTY_COALESCED_SOURCE_TABLE_ATTRIBUTE

      This attribute is applied to the descendants of source tables, and takes on Boolean values.

      • True for the result of coalesce() on source tables and their children if the source table was initially empty on coalesce.
      • Missing for all other tables.
      This effectively serves as a hint that filters may have been poorly selected, resulting in an empty result. If size() > 0, this hint should be disregarded.
      See Also:
      Constant Field Values

    • PREVIEW_PARENT_TABLE

      static final String PREVIEW_PARENT_TABLE
      This attribute stores a reference to a table that is the parent table for a Preview Table.
      See Also:
      Constant Field Values

    • NON_DISPLAY_TABLE

      static final String NON_DISPLAY_TABLE
      Set this attribute for tables that should not be displayed in the UI.
      See Also:
      Constant Field Values

    • PLUGIN_NAME

      static final String PLUGIN_NAME
      Set this attribute to load a plugin for this table in the Web Client
      See Also:
      Constant Field Values

    • BARRAGE_PERFORMANCE_KEY_ATTRIBUTE

      static final String BARRAGE_PERFORMANCE_KEY_ATTRIBUTE
      Set this attribute to enable collection of barrage performance stats.
      See Also:
      Constant Field Values

  • Method Details

    • getDefinition

      @ConcurrentMethod TableDefinition getDefinition()

    • getMeta

      @ConcurrentMethod Table getMeta()
      Provides column metadata in Table form. Convenience method, behaves exactly the same as getDefinition().getColumnDefinitionsTable().
      Returns:
      A Table of metadata about this Table's columns.

    • getDescription

      @ConcurrentMethod String getDescription()

    • numColumns

      @ConcurrentMethod int numColumns()
      Get the number of columns defined for this table. Equivalent to getDefinition().getColumns().length.
      Returns:
      The number of columns defined for this table

    • hasColumns

      @ConcurrentMethod boolean hasColumns​(String... columnNames)
      Determines whether this Table contains a column for each string in the specified array of columnNames.
      Parameters:
      columnNames - The array of column names to be checked for inclusion in this table. Must not be null.
      Returns:
      true if this Table contains a column for each and every string in the columnNames array; false if any element of columnNames is not the name of a column in this table

    • hasColumns

      @ConcurrentMethod boolean hasColumns​(Collection<String> columnNames)
      Determines whether this Table contains a column for each string in the specified collection of columnNames.
      Parameters:
      columnNames - The collection of column names to be checked for inclusion in this table. Must not be null.
      Returns:
      true if this Table contains a column for each and every string in the columnNames collection; false if any element of columnNames is not the name of a column in this table

    • isRefreshing

      @ConcurrentMethod boolean isRefreshing()
      Description copied from interface: DynamicNode
      Is the node updating?
      Specified by:
      isRefreshing in interface DynamicNode
      Returns:
      true if the node is updating; false otherwise.

    • getRowSet

      TrackingRowSet getRowSet()
      Returns:
      The TrackingRowSet that exposes the row keys present in this Table

    • sizeForInstrumentation

      long sizeForInstrumentation()
      Returns:
      Size if it is currently known without subsequent steps to coalesce the Table, else null

    • isEmpty

      boolean isEmpty()
      Returns true if this table has no rows (i.e. size() == 0).
      Returns:
      true if this table has no rows

    • isFlat

      @ConcurrentMethod boolean isFlat()
      Return true if this table is guaranteed to be flat. The RowSet of a flat table will be from 0...numRows-1.

    • getColumnSource

      <T> ColumnSource<T> getColumnSource​(String sourceName)
      Retrieves a ColumnSource. It is conveniently cast to @{code ColumnSource} using the type that caller expects. This differs from getColumnSource(String, Class) which uses the provided Class object to verify that the data type is a subclass of the expected class.
      Type Parameters:
      T - The target type, as a type parameter. Inferred from context.
      Parameters:
      sourceName - The name of the column
      Returns:
      The column source for sourceName, parameterized by T

    • getColumnSource

      <T> ColumnSource<T> getColumnSource​(String sourceName, Class<? extends T> clazz)
      Retrieves a ColumnSource and casts it to the target class clazz.
      Type Parameters:
      T - The target type, as a type parameter. Intended to be inferred from clazz.
      Parameters:
      sourceName - The name of the column
      clazz - The target type
      Returns:
      The column source for sourceName, parameterized by T

    • getColumnSourceMap

      Map<String,​? extends ColumnSource<?>> getColumnSourceMap()

    • getColumnSources

      Collection<? extends ColumnSource<?>> getColumnSources()

    • getColumns

      DataColumn[] getColumns()

    • getColumn

      DataColumn getColumn​(int columnIndex)

    • getColumn

      DataColumn getColumn​(String columnName)

    • columnIterator

      <TYPE> Iterator<TYPE> columnIterator​(@NotNull String columnName)

    • characterColumnIterator

      CharacterColumnIterator characterColumnIterator​(@NotNull String columnName)

    • byteColumnIterator

      ByteColumnIterator byteColumnIterator​(@NotNull String columnName)

    • shortColumnIterator

      ShortColumnIterator shortColumnIterator​(@NotNull String columnName)

    • integerColumnIterator

      IntegerColumnIterator integerColumnIterator​(@NotNull String columnName)

    • longColumnIterator

      LongColumnIterator longColumnIterator​(@NotNull String columnName)

    • floatColumnIterator

      FloatColumnIterator floatColumnIterator​(@NotNull String columnName)

    • doubleColumnIterator

      DoubleColumnIterator doubleColumnIterator​(@NotNull String columnName)

    • objectColumnIterator

      <DATA_TYPE> ObjectColumnIterator<DATA_TYPE> objectColumnIterator​(@NotNull String columnName)

    • getRecord

      Object[] getRecord​(long rowNo, String... columnNames)

    • where

      @ConcurrentMethod Table where​(Filter... filters)

    • wouldMatch

      @ConcurrentMethod Table wouldMatch​(WouldMatchPair... matchers)
      A table operation that applies the supplied predicate to each row in the table and produces columns containing the pass/fail result of the predicate application. This is similar to TableOperations.where(String...) except that instead of selecting only rows that meet the criteria, new columns are added with the result of the comparison.
      Returns:
      a table with new columns containing the filter result for each row.

    • wouldMatch

      @ConcurrentMethod Table wouldMatch​(String... expressions)

    • select

      Table select​(Selectable... columns)

    • select

      Table select()

    • update

      Table update​(Selectable... newColumns)

    • lazyUpdate

      Table lazyUpdate​(Selectable... newColumns)

    • view

      @ConcurrentMethod Table view​(Selectable... columns)

    • updateView

      @ConcurrentMethod Table updateView​(Selectable... newColumns)

    • dropColumns

      @ConcurrentMethod Table dropColumns​(Collection<String> columnNames)

    • dropColumns

      @ConcurrentMethod Table dropColumns​(String... columnNames)

    • dropColumnFormats

      @ConcurrentMethod Table dropColumnFormats()

    • renameColumns

      Table renameColumns​(MatchPair... pairs)

    • renameColumns

      Table renameColumns​(Collection<String> columns)

    • renameColumns

      Table renameColumns​(String... columns)

    • renameAllColumns

      Table renameAllColumns​(UnaryOperator<String> renameFunction)

    • formatColumns

      @ConcurrentMethod Table formatColumns​(String... columnFormats)

    • formatRowWhere

      @ConcurrentMethod Table formatRowWhere​(String condition, String formula)

    • formatColumnWhere

      @ConcurrentMethod Table formatColumnWhere​(String columnName, String condition, String formula)

    • moveColumnsUp

      @ConcurrentMethod Table moveColumnsUp​(String... columnsToMove)
      Produce a new table with the specified columns moved to the leftmost position. Columns can be renamed with the usual syntax, i.e. "NewColumnName=OldColumnName").
      Parameters:
      columnsToMove - The columns to move to the left (and, optionally, to rename)
      Returns:
      The new table, with the columns rearranged as explained above moveColumns(int, String...)

    • moveColumnsDown

      @ConcurrentMethod Table moveColumnsDown​(String... columnsToMove)
      Produce a new table with the specified columns moved to the rightmost position. Columns can be renamed with the usual syntax, i.e. "NewColumnName=OldColumnName").
      Parameters:
      columnsToMove - The columns to move to the right (and, optionally, to rename)
      Returns:
      The new table, with the columns rearranged as explained above moveColumns(int, String...)

    • moveColumns

      @ConcurrentMethod Table moveColumns​(int index, String... columnsToMove)
      Produce a new table with the specified columns moved to the specified index. Column indices begin at 0. Columns can be renamed with the usual syntax, i.e. "NewColumnName=OldColumnName").
      Parameters:
      index - The index to which the specified columns should be moved
      columnsToMove - The columns to move to the specified index (and, optionally, to rename)
      Returns:
      The new table, with the columns rearranged as explained above

    • moveColumns

      @ConcurrentMethod Table moveColumns​(int index, boolean moveToEnd, String... columnsToMove)

    • dateTimeColumnAsNanos

      @ConcurrentMethod Table dateTimeColumnAsNanos​(String dateTimeColumnName, String nanosColumnName)
      Produce a new table with the same columns as this table, but with a new column presenting the specified DateTime column as a Long column (with each DateTime represented instead as the corresponding number of nanos since the epoch).

      NOTE: This is a really just an updateView(), and behaves accordingly for column ordering and (re)placement. This doesn't work on data that has been brought fully into memory (e.g. via select()). Use a view instead.

      Parameters:
      dateTimeColumnName - Name of date time column
      nanosColumnName - Name of nanos column
      Returns:
      The new table, constructed as explained above.

    • dateTimeColumnAsNanos

      @ConcurrentMethod Table dateTimeColumnAsNanos​(String columnName)
      Parameters:
      columnName - name of column to convert from DateTime to nanos
      Returns:
      The result of dateTimeColumnAsNanos(columnName, columnName).

    • slice

      @ConcurrentMethod Table slice​(long firstPositionInclusive, long lastPositionExclusive)
      Extracts a subset of a table by row position.

      If both firstPosition and lastPosition are positive, then the rows are counted from the beginning of the table. The firstPosition is inclusive, and the lastPosition is exclusive. The TableOperations.head(long)(N) call is equivalent to slice(0, N). The firstPosition must be less than or equal to the lastPosition.

      If firstPosition is positive and lastPosition is negative, then the firstRow is counted from the beginning of the table, inclusively. The lastPosition is counted from the end of the table. For example, slice(1, -1) includes all rows but the first and last. If the lastPosition would be before the firstRow, the result is an emptyTable.

      If firstPosition is negative, and lastPosition is zero, then the firstRow is counted from the end of the table, and the end of the slice is the size of the table. slice(-N, 0) is equivalent to TableOperations.tail(long)(N).

      If the firstPosition is negative and the lastPosition is negative, they are both counted from the end of the table. For example, slice(-2, -1) returns the second to last row of the table.

      Parameters:
      firstPositionInclusive - the first position to include in the result
      lastPositionExclusive - the last position to include in the result
      Returns:
      a new Table, which is the request subset of rows from the original table

    • headPct

      @ConcurrentMethod Table headPct​(double percent)
      Provides a head that selects a dynamic number of rows based on a percent.
      Parameters:
      percent - the fraction of the table to return (0..1), the number of rows will be rounded up. For example if there are 3 rows, headPct(50) returns the first two rows.

    • tailPct

      @ConcurrentMethod Table tailPct​(double percent)

    • exactJoin

      Table exactJoin​(Table rightTable, MatchPair[] columnsToMatch, MatchPair[] columnsToAdd)

    • aj

      Table aj​(Table rightTable, MatchPair[] columnsToMatch, MatchPair[] columnsToAdd, Table.AsOfMatchRule asOfMatchRule)
      Looks up the columns in the rightTable that meet the match conditions in the columnsToMatch list. Matching is done exactly for the first n-1 columns and via a binary search for the last match pair. The columns of the original table are returned intact, together with the columns from rightTable defined in a comma separated list "columnsToAdd"
      Parameters:
      rightTable - The right side table on the join.
      columnsToMatch - A comma separated list of match conditions ("leftColumn=rightColumn" or "columnFoundInBoth")
      columnsToAdd - A comma separated list with the columns from the left side that need to be added to the right side as a result of the match.
      Returns:
      a new table joined according to the specification in columnsToMatch and columnsToAdd

    • aj

      Table aj​(Table rightTable, MatchPair[] columnsToMatch, MatchPair[] columnsToAdd)
      Looks up the columns in the rightTable that meet the match conditions in the columnsToMatch list. Matching is done exactly for the first n-1 columns and via a binary search for the last match pair. The columns of the original table are returned intact, together with the columns from rightTable defined in a comma separated list "columnsToAdd"
      Parameters:
      rightTable - The right side table on the join.
      columnsToMatch - A comma separated list of match conditions ("leftColumn=rightColumn" or "columnFoundInBoth")
      columnsToAdd - A comma separated list with the columns from the left side that need to be added to the right side as a result of the match.
      Returns:
      a new table joined according to the specification in columnsToMatch and columnsToAdd

    • aj

      Table aj​(Table rightTable, Collection<String> columnsToMatch)
      Looks up the columns in the rightTable that meet the match conditions in the columnsToMatch list. Matching is done exactly for the first n-1 columns and via a binary search for the last match pair. The columns of the original table are returned intact, together with all the columns from rightTable.
      Parameters:
      rightTable - The right side table on the join.
      columnsToMatch - A comma separated list of match conditions ("leftColumn=rightColumn" or "columnFoundInBoth")
      Returns:
      a new table joined according to the specification in columnsToMatch and columnsToAdd

    • raj

      Table raj​(Table rightTable, MatchPair[] columnsToMatch, MatchPair[] columnsToAdd, Table.AsOfMatchRule asOfMatchRule)
      Just like .aj(), but the matching on the last column is in reverse order, so that you find the row after the given timestamp instead of the row before.

      Looks up the columns in the rightTable that meet the match conditions in the columnsToMatch list. Matching is done exactly for the first n-1 columns and via a binary search for the last match pair. The columns of the original table are returned intact, together with the columns from rightTable defined in a comma separated list "columnsToAdd"

      Parameters:
      rightTable - The right side table on the join.
      columnsToMatch - A comma separated list of match conditions ("leftColumn=rightColumn" or "columnFoundInBoth")
      columnsToAdd - A comma separated list with the columns from the left side that need to be added to the right side as a result of the match.
      Returns:
      a new table joined according to the specification in columnsToMatch and columnsToAdd

    • raj

      Table raj​(Table rightTable, MatchPair[] columnsToMatch, MatchPair[] columnsToAdd)
      Just like .aj(), but the matching on the last column is in reverse order, so that you find the row after the given timestamp instead of the row before.

      Looks up the columns in the rightTable that meet the match conditions in the columnsToMatch list. Matching is done exactly for the first n-1 columns and via a binary search for the last match pair. The columns of the original table are returned intact, together with the columns from rightTable defined in a comma separated list "columnsToAdd"

      Parameters:
      rightTable - The right side table on the join.
      columnsToMatch - A comma separated list of match conditions ("leftColumn=rightColumn" or "columnFoundInBoth")
      columnsToAdd - A comma separated list with the columns from the left side that need to be added to the right side as a result of the match.
      Returns:
      a new table joined according to the specification in columnsToMatch and columnsToAdd

    • raj

      Table raj​(Table rightTable, Collection<String> columnsToMatch)
      Just like .aj(), but the matching on the last column is in reverse order, so that you find the row after the given timestamp instead of the row before.

      Looks up the columns in the rightTable that meet the match conditions in the columnsToMatch list. Matching is done exactly for the first n-1 columns and via a binary search for the last match pair. The columns of the original table are returned intact, together with the all columns from rightTable.

      Parameters:
      rightTable - The right side table on the join.
      columnsToMatch - A comma separated list of match conditions ("leftColumn=rightColumn" or "columnFoundInBoth")
      Returns:
      a new table joined according to the specification in columnsToMatch and columnsToAdd

    • naturalJoin

      Table naturalJoin​(Table rightTable, MatchPair[] columnsToMatch, MatchPair[] columnsToAdd)

    • join

      Table join​(Table rightTable)
      Perform a cross join with the right table.

      Returns a table that is the cartesian product of left rows X right rows, with one column for each of the left table's columns, and one column corresponding to each of the right table's columns. The rows are ordered first by the left table then by the right table.

      To efficiently produce updates, the bits that represent a key for a given row are split into two. Unless specified, join reserves 16 bits to represent a right row. When there are too few bits to represent all of the right rows for a given aggregation group the table will shift a bit from the left side to the right side. The default of 16 bits was carefully chosen because it results in an efficient implementation to process live updates.

      An OutOfKeySpaceException is thrown when the total number of bits needed to express the result table exceeds that needed to represent Long.MAX_VALUE. There are a few work arounds: - If the left table is sparse, consider flattening the left table. - If there are no key-columns and the right table is sparse, consider flattening the right table. - If the maximum size of a right table's group is small, you can reserve fewer bits by setting numRightBitsToReserve on initialization.

      Note: If you can prove that a given group has at most one right-row then you should prefer using naturalJoin(io.deephaven.engine.table.Table, io.deephaven.engine.table.MatchPair[], io.deephaven.engine.table.MatchPair[]).

      Parameters:
      rightTable - The right side table on the join.
      Returns:
      a new table joined according to the specification with zero key-columns and includes all right columns

    • join

      Table join​(Table rightTable, int numRightBitsToReserve)
      Perform a cross join with the right table.

      Returns a table that is the cartesian product of left rows X right rows, with one column for each of the left table's columns, and one column corresponding to each of the right table's columns. The rows are ordered first by the left table then by the right table.

      To efficiently produce updates, the bits that represent a key for a given row are split into two. Unless specified, join reserves 16 bits to represent a right row. When there are too few bits to represent all of the right rows for a given aggregation group the table will shift a bit from the left side to the right side. The default of 16 bits was carefully chosen because it results in an efficient implementation to process live updates.

      An OutOfKeySpaceException is thrown when the total number of bits needed to express the result table exceeds that needed to represent Long.MAX_VALUE. There are a few work arounds: - If the left table is sparse, consider flattening the left table. - If there are no key-columns and the right table is sparse, consider flattening the right table. - If the maximum size of a right table's group is small, you can reserve fewer bits by setting numRightBitsToReserve on initialization.

      Note: If you can prove that a given group has at most one right-row then you should prefer using naturalJoin(io.deephaven.engine.table.Table, io.deephaven.engine.table.MatchPair[], io.deephaven.engine.table.MatchPair[]).

      Parameters:
      rightTable - The right side table on the join.
      numRightBitsToReserve - The number of bits to reserve for rightTable groups.
      Returns:
      a new table joined according to the specification with zero key-columns and includes all right columns

    • join

      Table join​(Table rightTable, String columnsToMatch, int numRightBitsToReserve)
      Perform a cross join with the right table.

      Returns a table that is the cartesian product of left rows X right rows, with one column for each of the left table's columns, and one column corresponding to each of the right table's columns that are not key-columns. The rows are ordered first by the left table then by the right table. If columnsToMatch is non-empty then the product is filtered by the supplied match conditions.

      To efficiently produce updates, the bits that represent a key for a given row are split into two. Unless specified, join reserves 16 bits to represent a right row. When there are too few bits to represent all of the right rows for a given aggregation group the table will shift a bit from the left side to the right side. The default of 16 bits was carefully chosen because it results in an efficient implementation to process live updates.

      An OutOfKeySpaceException is thrown when the total number of bits needed to express the result table exceeds that needed to represent Long.MAX_VALUE. There are a few work arounds: - If the left table is sparse, consider flattening the left table. - If there are no key-columns and the right table is sparse, consider flattening the right table. - If the maximum size of a right table's group is small, you can reserve fewer bits by setting numRightBitsToReserve on initialization.

      Note: If you can prove that a given group has at most one right-row then you should prefer using naturalJoin(io.deephaven.engine.table.Table, io.deephaven.engine.table.MatchPair[], io.deephaven.engine.table.MatchPair[]).

      Parameters:
      rightTable - The right side table on the join.
      columnsToMatch - A comma separated list of match conditions ("leftColumn=rightColumn" or "columnFoundInBoth")
      numRightBitsToReserve - The number of bits to reserve for rightTable groups.
      Returns:
      a new table joined according to the specification in columnsToMatch and includes all non-key-columns from the right table

    • join

      Table join​(Table rightTable, String columnsToMatch, String columnsToAdd, int numRightBitsToReserve)
      Perform a cross join with the right table.

      Returns a table that is the cartesian product of left rows X right rows, with one column for each of the left table's columns, and one column corresponding to each of the right table's columns that are included in the columnsToAdd argument. The rows are ordered first by the left table then by the right table. If columnsToMatch is non-empty then the product is filtered by the supplied match conditions.

      To efficiently produce updates, the bits that represent a key for a given row are split into two. Unless specified, join reserves 16 bits to represent a right row. When there are too few bits to represent all of the right rows for a given aggregation group the table will shift a bit from the left side to the right side. The default of 16 bits was carefully chosen because it results in an efficient implementation to process live updates.

      An OutOfKeySpaceException is thrown when the total number of bits needed to express the result table exceeds that needed to represent Long.MAX_VALUE. There are a few work arounds: - If the left table is sparse, consider flattening the left table. - If there are no key-columns and the right table is sparse, consider flattening the right table. - If the maximum size of a right table's group is small, you can reserve fewer bits by setting numRightBitsToReserve on initialization.

      Note: If you can prove that a given group has at most one right-row then you should prefer using naturalJoin(io.deephaven.engine.table.Table, io.deephaven.engine.table.MatchPair[], io.deephaven.engine.table.MatchPair[]).

      Parameters:
      rightTable - The right side table on the join.
      columnsToMatch - A comma separated list of match conditions ("leftColumn=rightColumn" or "columnFoundInBoth")
      columnsToAdd - A comma separated list with the columns from the right side that need to be added to the left side as a result of the match.
      numRightBitsToReserve - The number of bits to reserve for rightTable groups.
      Returns:
      a new table joined according to the specification in columnsToMatch and columnsToAdd

    • join

      Table join​(Table rightTable, MatchPair[] columnsToMatch, MatchPair[] columnsToAdd)
      Perform a cross join with the right table.

      Returns a table that is the cartesian product of left rows X right rows, with one column for each of the left table's columns, and one column corresponding to each of the right table's columns that are included in the columnsToAdd argument. The rows are ordered first by the left table then by the right table. If columnsToMatch is non-empty then the product is filtered by the supplied match conditions.

      To efficiently produce updates, the bits that represent a key for a given row are split into two. Unless specified, join reserves 16 bits to represent a right row. When there are too few bits to represent all of the right rows for a given aggregation group the table will shift a bit from the left side to the right side. The default of 16 bits was carefully chosen because it results in an efficient implementation to process live updates.

      An OutOfKeySpaceException is thrown when the total number of bits needed to express the result table exceeds that needed to represent Long.MAX_VALUE. There are a few work arounds: - If the left table is sparse, consider flattening the left table. - If there are no key-columns and the right table is sparse, consider flattening the right table. - If the maximum size of a right table's group is small, you can reserve fewer bits by setting numRightBitsToReserve on initialization.

      Note: If you can prove that a given group has at most one right-row then you should prefer using naturalJoin(io.deephaven.engine.table.Table, io.deephaven.engine.table.MatchPair[], io.deephaven.engine.table.MatchPair[]).

      Parameters:
      rightTable - The right side table on the join.
      columnsToMatch - An array of match pair conditions ("leftColumn=rightColumn" or "columnFoundInBoth")
      columnsToAdd - An array of the columns from the right side that need to be added to the left side as a result of the match.
      Returns:
      a new table joined according to the specification in columnsToMatch and columnsToAdd

    • join

      Table join​(Table rightTable, MatchPair[] columnsToMatch, MatchPair[] columnsToAdd, int numRightBitsToReserve)
      Perform a cross join with the right table.

      Returns a table that is the cartesian product of left rows X right rows, with one column for each of the left table's columns, and one column corresponding to each of the right table's columns that are included in the columnsToAdd argument. The rows are ordered first by the left table then by the right table. If columnsToMatch is non-empty then the product is filtered by the supplied match conditions.

      To efficiently produce updates, the bits that represent a key for a given row are split into two. Unless specified, join reserves 16 bits to represent a right row. When there are too few bits to represent all of the right rows for a given aggregation group the table will shift a bit from the left side to the right side. The default of 16 bits was carefully chosen because it results in an efficient implementation to process live updates.

      An OutOfKeySpaceException is thrown when the total number of bits needed to express the result table exceeds that needed to represent Long.MAX_VALUE. There are a few work arounds: - If the left table is sparse, consider flattening the left table. - If there are no key-columns and the right table is sparse, consider flattening the right table. - If the maximum size of a right table's group is small, you can reserve fewer bits by setting numRightBitsToReserve on initialization.

      Note: If you can prove that a given group has at most one right-row then you should prefer using naturalJoin(io.deephaven.engine.table.Table, io.deephaven.engine.table.MatchPair[], io.deephaven.engine.table.MatchPair[]).

      Parameters:
      rightTable - The right side table on the join.
      columnsToMatch - An array of match pair conditions ("leftColumn=rightColumn" or "columnFoundInBoth")
      columnsToAdd - An array of the columns from the right side that need to be added to the left side as a result of the match.
      numRightBitsToReserve - The number of bits to reserve for rightTable groups.
      Returns:
      a new table joined according to the specification in columnsToMatch and columnsToAdd

    • headBy

      Table headBy​(long nRows, Collection<String> groupByColumnNames)

    • headBy

      Table headBy​(long nRows, String... groupByColumnNames)

    • tailBy

      Table tailBy​(long nRows, Collection<String> groupByColumnNames)

    • tailBy

      Table tailBy​(long nRows, String... groupByColumnNames)

    • applyToAllBy

      @ConcurrentMethod Table applyToAllBy​(String formulaColumn, String columnParamName, Collection<? extends ColumnName> groupByColumns)
      Groups data according to groupByColumns and applies formulaColumn to each of columns not altered by the grouping operation. columnParamName is used as place-holder for the name of each column inside formulaColumn.
      Parameters:
      formulaColumn - Formula applied to each column
      columnParamName - The parameter name used as a placeholder for each column
      groupByColumns - The grouping columns as in TableOperations.groupBy(Collection)

    • applyToAllBy

      @ConcurrentMethod Table applyToAllBy​(String formulaColumn, Collection<? extends ColumnName> groupByColumns)
      Groups data according to groupByColumns and applies formulaColumn to each of columns not altered by the grouping operation.
      Parameters:
      formulaColumn - Formula applied to each column, uses parameter each to refer to each colum it being applied to
      groupByColumns - The grouping columns as in TableOperations.groupBy(Collection)

    • applyToAllBy

      @ConcurrentMethod Table applyToAllBy​(String formulaColumn, String... groupByColumns)
      Groups data according to groupByColumns and applies formulaColumn to each of columns not altered by the grouping operation.
      Parameters:
      formulaColumn - Formula applied to each column, uses parameter each to refer to each colum it being applied to
      groupByColumns - The grouping columns as in TableOperations.groupBy()

    • dropStream

      @ConcurrentMethod Table dropStream()
      If this table is a stream table, i.e. it has STREAM_TABLE_ATTRIBUTE set to true, return a child without the attribute, restoring standard semantics for aggregation operations.
      Returns:
      A non-stream child table, or this table if it is not a stream table

    • ungroupAllBut

      Table ungroupAllBut​(String... columnsNotToUngroup)

    • partitionBy

      @ConcurrentMethod PartitionedTable partitionBy​(boolean dropKeys, String... keyColumnNames)
      Create a PartitionedTable from this table, partitioned according to the specified key columns.

      The underlying partitioned table backing the result contains each row in this table in exactly one of the result's constituent tables.

      Parameters:
      dropKeys - Whether to drop key columns in the output constituent tables
      keyColumnNames - The name of the key columns to partition by
      Returns:
      A PartitionedTable keyed by keyColumnNames

    • partitionBy

      @ConcurrentMethod PartitionedTable partitionBy​(String... keyColumnNames)
      Equivalent to partitionBy(false, keyColumnNames)

      Create a PartitionedTable from this table, partitioned according to the specified key columns. Key columns are never dropped from the output constituent tables.

      The underlying partitioned table backing the result contains each row in this table in exactly one of the result's constituent tables.

      Parameters:
      keyColumnNames - The name of the key columns to partition by
      Returns:
      A PartitionedTable keyed by keyColumnNames

    • partitionedAggBy

      @ConcurrentMethod PartitionedTable partitionedAggBy​(Collection<? extends Aggregation> aggregations, boolean preserveEmpty, Table initialGroups, String... keyColumnNames)
      Convenience method that performs an TableOperations.aggBy(io.deephaven.api.agg.Aggregation) and wraps the result in a PartitionedTable. If aggregations does not include a partition, one will be added automatically with the default constituent column name and behavior used in partitionBy(String...).
      Parameters:
      aggregations - The aggregations to apply
      preserveEmpty - Whether to keep result rows for groups that are initially empty or become empty as a result of updates. Each aggregation operator defines its own value for empty groups.
      initialGroups - A table whose distinct combinations of values for the groupByColumns should be used to create an initial set of aggregation groups. All other columns are ignored. This is useful in combination with preserveEmpty == true to ensure that particular groups appear in the result table, or with preserveEmpty == false to control the encounter order for a collection of groups and thus their relative order in the result. Changes to initialGroups are not expected or handled; if initialGroups is a refreshing table, only its contents at instantiation time will be used. If initialGroups == null, the result will be the same as if a table with no rows was supplied.
      keyColumnNames - The names of the key columns to aggregate by
      Returns:
      A PartitionedTable keyed by keyColumnNames

    • rollup

      @ConcurrentMethod RollupTable rollup​(Collection<? extends Aggregation> aggregations)
      Create a rollup table.

      A rollup table aggregates all rows of the table.

      Parameters:
      aggregations - The aggregations to perform
      Returns:
      a hierarchical table with the rollup applied

    • rollup

      @ConcurrentMethod RollupTable rollup​(Collection<? extends Aggregation> aggregations, boolean includeConstituents)
      Create a rollup table.

      A rollup table aggregates all rows of the table.

      Parameters:
      aggregations - The aggregations to perform
      includeConstituents - set to true to include the constituent rows at the leaf level
      Returns:
      a hierarchical table with the rollup applied

    • rollup

      @ConcurrentMethod RollupTable rollup​(Collection<? extends Aggregation> aggregations, String... groupByColumns)
      Create a rollup table.

      A rollup table aggregates by the specified columns, and then creates a hierarchical table which re-aggregates using one less aggregation column on each level. The column that is no longer part of the aggregation key is replaced with null on each level.

      Parameters:
      aggregations - The aggregations to perform
      groupByColumns - the columns to group by
      Returns:
      a hierarchical table with the rollup applied

    • rollup

      @ConcurrentMethod RollupTable rollup​(Collection<? extends Aggregation> aggregations, boolean includeConstituents, String... groupByColumns)
      Create a rollup table.

      A rollup table aggregates by the specified columns, and then creates a hierarchical table which re-aggregates using one less aggregation column on each level. The column that is no longer part of the aggregation key is replaced with null on each level.

      Parameters:
      aggregations - The aggregations to perform
      groupByColumns - the columns to group by
      includeConstituents - set to true to include the constituent rows at the leaf level
      Returns:
      a hierarchical table with the rollup applied

    • rollup

      @ConcurrentMethod RollupTable rollup​(Collection<? extends Aggregation> aggregations, Collection<? extends ColumnName> groupByColumns)
      Create a rollup table.

      A rollup table aggregates by the specified columns, and then creates a hierarchical table which re-aggregates using one less aggregation column on each level. The column that is no longer part of the aggregation key is replaced with null on each level.

      Parameters:
      aggregations - The aggregations to perform
      groupByColumns - the columns to group by
      Returns:
      a hierarchical table with the rollup applied

    • rollup

      @ConcurrentMethod RollupTable rollup​(Collection<? extends Aggregation> aggregations, boolean includeConstituents, Collection<? extends ColumnName> groupByColumns)
      Create a rollup table.

      A rollup table aggregates by the specified columns, and then creates a hierarchical table which re-aggregates using one less aggregation column on each level. The column that is no longer part of the aggregation key is replaced with null on each level.

      Parameters:
      aggregations - The aggregations to perform
      includeConstituents - set to true to include the constituent rows at the leaf level
      groupByColumns - the columns to group by
      Returns:
      a hierarchical table with the rollup applied

    • tree

      @ConcurrentMethod TreeTable tree​(String idColumn, String parentColumn)
      Create a hierarchical tree table.

      The structure of the table is encoded by an "id" and a "parent" column. The id column should represent a unique identifier for a given row, and the parent column indicates which row is the parent for a given row. Rows that have a null parent are part of the "root" table.

      It is possible for rows to be "orphaned" if their parent is non-null and does not exist in the table. See TreeTable.promoteOrphans(Table, String, String).

      Parameters:
      idColumn - The name of a column containing a unique identifier for a particular row in the table
      parentColumn - The name of a column containing the parent's identifier, null for rows that are part of the root table
      Returns:
      A TreeTable organized according to the parent-child relationships expressed by idColumn and parentColumn

    • mergeBefore

      Table mergeBefore​(Table... others)
      Merge this Table with others. All rows in this Table will appear before all rows in others. If Tables in others are the result of a prior merge operation, they may be expanded in an attempt to avoid deeply nested structures.
      Parameters:
      others - The Tables to merge with
      Returns:
      The merged Table
      ApiNote:
      It's best to avoid many chained calls to mergeBefore(Table...) and mergeAfter(Table...), as this may result in deeply-nested data structures. See TableTools.merge(Table...).

    • mergeAfter

      Table mergeAfter​(Table... others)
      Merge this Table with others. All rows in this Table will appear after all rows in others. If Tables in others are the result of a prior merge operation, they may be expanded in an attempt to avoid deeply nested structures.
      Parameters:
      others - The Tables to merge with
      Returns:
      The merged Table
      ApiNote:
      It's best to avoid many chained calls to mergeBefore(Table...) and mergeAfter(Table...), as this may result in deeply-nested data structures. See TableTools.merge(Table...).

    • coalesce

      @ConcurrentMethod Table coalesce()
      Explicitly ensure that any work needed to make a table addressable, iterable, or queryable has been done, and return the coalesced child table if appropriate.
      Returns:
      This table, or a fully-coalesced child

    • getSubTable

      Table getSubTable​(TrackingRowSet rowSet)
      Get a Table that contains a sub-set of the rows from this. The result will share the same column sources and definition as this table. The result will not update on its own. The caller must also establish an appropriate listener to update rowSet and propagate updates.
      Parameters:
      rowSet - The row set for the result
      Returns:
      A new sub-table

    • apply

      <R> R apply​(Function<Table,​R> function)
      Applies a function to this table.

      This is useful if you have a reference to a table or a proxy and want to run a series of operations against the table without each individual operation resulting in an RMI.

      Type Parameters:
      R - the return type of function
      Parameters:
      function - the function to run, its single argument will be this table
      Returns:
      the return value of function
      ImplNote:
      If the UGP is not required the Function.apply(Object) method should be annotated with ConcurrentMethod.

    • flatten

      @ConcurrentMethod Table flatten()
      Creates a version of this table with a flat RowSet.

    • withKeys

      @ConcurrentMethod Table withKeys​(String... columns)
      Set the table's key columns.
      Returns:
      A copy of this table with the key columns specified, or this if no change was needed

    • withUniqueKeys

      @ConcurrentMethod Table withUniqueKeys​(String... columns)
      Set the table's key columns and indicate that each key set will be unique.
      Returns:
      A copy of this table with the unique key columns specified, or this if no change was needed

    • setTotalsTable

      @ConcurrentMethod Table setTotalsTable​(String directive)
      Set a totals table for this Table.
      Parameters:
      directive - A packed string of totals table instructions
      Returns:
      A copy of this Table with the totals table attribute set

    • close

      void close()
      Release resources held by this table, possibly destructively. This may render the table unsuitable or unsafe for further use.
      ApiNote:
      In practice, implementations usually just invoke releaseCachedResources().

    • releaseCachedResources

      void releaseCachedResources()
      Attempt to release cached resources held by this table. Unlike close(), this must not render the table unusable for subsequent read operations. Implementations should be sure to call super.releaseCachedResources().

    • awaitUpdate

      void awaitUpdate() throws InterruptedException

      Wait for updates to this Table.

      In some implementations, this call may also terminate in case of interrupt or spurious wakeup (see java.util.concurrent.locks.Condition#await()).

      Throws:
      InterruptedException - In the event this thread is interrupted

    • awaitUpdate

      boolean awaitUpdate​(long timeout) throws InterruptedException

      Wait for updates to this Table.

      In some implementations, this call may also terminate in case of interrupt or spurious wakeup (see java.util.concurrent.locks.Condition#await()).

      Parameters:
      timeout - The maximum time to wait in milliseconds.
      Returns:
      false if the timeout elapses without notification, true otherwise.
      Throws:
      InterruptedException - In the event this thread is interrupted

    • addUpdateListener

      void addUpdateListener​(ShiftObliviousListener listener)
      Subscribe for updates to this table. listener will be invoked via the NotificationQueue associated with this Table.
      Parameters:
      listener - listener for updates

    • addUpdateListener

      void addUpdateListener​(ShiftObliviousListener listener, boolean replayInitialImage)
      Subscribe for updates to this table. After the optional initial image, listener will be invoked via the NotificationQueue associated with this Table.
      Parameters:
      listener - listener for updates
      replayInitialImage - true to process updates for all initial rows in the table plus all changes; false to only process changes

    • addUpdateListener

      void addUpdateListener​(TableUpdateListener listener)
      Subscribe for updates to this table. listener will be invoked via the NotificationQueue associated with this Table.
      Parameters:
      listener - listener for updates

    • removeUpdateListener

      void removeUpdateListener​(ShiftObliviousListener listener)
      Unsubscribe the supplied listener.
      Parameters:
      listener - listener for updates

    • removeUpdateListener

      void removeUpdateListener​(TableUpdateListener listener)
      Unsubscribe the supplied listener.
      Parameters:
      listener - listener for updates

    • isFailed

      boolean isFailed()
      Returns:
      true if this table is in a failure state.