Class UncoalescedTable<IMPL_TYPE extends UncoalescedTable<IMPL_TYPE>>
- All Implemented Interfaces:
TableOperations<Table,
,Table> TableOperationsDefaults<Table,
,Table> LogOutputAppendable
,LivenessManager
,LivenessNode
,LivenessReferent
,AttributeMap<Table>
,GridAttributes<Table>
,NotificationStepReceiver
,NotificationStepSource
,TableDefaults
,Table
,DynamicNode
,NotificationQueue.Dependency
,SystemicObject<Table>
,LongSizedDataStructure
,Serializable
- Direct Known Subclasses:
RedefinableTable
- See Also:
-
Nested Class Summary
Nested classes/interfaces inherited from class io.deephaven.engine.table.impl.BaseTable
BaseTable.CopyAttributeOperation, BaseTable.ListenerImpl, BaseTable.ShiftObliviousListenerImpl, BaseTable.SnapshotControlFactory<T extends ConstructSnapshot.SnapshotControl>
-
Field Summary
Fields inherited from class io.deephaven.engine.table.impl.BaseTable
definition, description, PRINT_SERIALIZED_UPDATE_OVERLAPS, updateGraph, VALIDATE_UPDATE_OVERLAPS
Fields inherited from interface io.deephaven.engine.table.GridAttributes
COLUMN_DESCRIPTIONS_ATTRIBUTE, DESCRIPTION_ATTRIBUTE, LAYOUT_HINTS_ATTRIBUTE, SORTABLE_COLUMNS_ATTRIBUTE
Fields inherited from interface io.deephaven.engine.table.impl.NotificationStepReceiver
NULL_NOTIFICATION_STEP
Fields inherited from interface io.deephaven.engine.table.Table
ADD_ONLY_TABLE_ATTRIBUTE, AGGREGATION_ROW_LOOKUP_ATTRIBUTE, APPEND_ONLY_TABLE_ATTRIBUTE, BARRAGE_PERFORMANCE_KEY_ATTRIBUTE, BLINK_TABLE_ATTRIBUTE, FILTERABLE_COLUMNS_ATTRIBUTE, INITIALLY_EMPTY_COALESCED_SOURCE_TABLE_ATTRIBUTE, INPUT_TABLE_ATTRIBUTE, KEY_COLUMNS_ATTRIBUTE, MERGED_TABLE_ATTRIBUTE, NON_DISPLAY_TABLE, PLUGIN_NAME, PREVIEW_PARENT_TABLE, SNAPSHOT_VIEWPORT_TYPE, SORT_REVERSE_LOOKUP_ATTRIBUTE, SORTED_COLUMNS_ATTRIBUTE, SYSTEMIC_TABLE_ATTRIBUTE, TEST_SOURCE_TABLE_ATTRIBUTE, TOTALS_TABLE_ATTRIBUTE, UNIQUE_KEYS_ATTRIBUTE
Fields inherited from interface io.deephaven.engine.table.impl.TableDefaults
ZERO_LENGTH_TABLE_ARRAY
Fields inherited from interface io.deephaven.api.TableOperations
AGG_BY_PRESERVE_EMPTY_DEFAULT
Fields inherited from interface io.deephaven.api.TableOperationsDefaults
ZERO_LENGTH_COLUMNNAME_ARRAY, ZERO_LENGTH_FILTER_ARRAY
-
Constructor Summary
ConstructorDescriptionUncoalescedTable
(@NotNull TableDefinition definition, @NotNull String description) -
Method Summary
Modifier and TypeMethodDescriptionvoid
addUpdateListener
(@NotNull TableUpdateListener listener) Subscribe for updates to this table.boolean
addUpdateListener
(@NotNull TableUpdateListener listener, long requiredLastNotificationStep) Subscribe for updates to this table if its last notification step matchesrequiredLastNotificationStep
.void
addUpdateListener
(ShiftObliviousListener listener, boolean replayInitialImage) Subscribe for updates to this table.protected final void
addUpdateListenerUncoalesced
(@NotNull TableUpdateListener listener) protected final boolean
addUpdateListenerUncoalesced
(@NotNull TableUpdateListener listener, long requiredLastNotificationStep) aggAllBy
(AggSpec spec, ColumnName... groupByColumns) aggBy
(Collection<? extends Aggregation> aggregations, boolean preserveEmpty, Table initialGroups, Collection<? extends ColumnName> groupByColumns) Produce an aggregated result by groupingthis
according to thegroupByColumns
and applyingaggregations
to each resulting group of rows.<R> R
Applies a function to this table.asOfJoin
(Table rightTable, Collection<? extends JoinMatch> exactMatches, AsOfJoinMatch asOfMatch, Collection<? extends JoinAddition> columnsToAdd) void
Wait for updates to this Table.boolean
awaitUpdate
(long timeout) Wait for updates to this Table.byteColumnIterator
(@NotNull String columnName) characterColumnIterator
(@NotNull String columnName) final Table
coalesce()
Explicitly ensure that any work needed to make a table addressable, iterable, or queryable has been done, and return the coalesced child table if appropriate.<TYPE> CloseableIterator<TYPE>
columnIterator
(@NotNull String columnName) protected abstract Table
Produce the actual coalesced result table, suitable for caching.doubleColumnIterator
(@NotNull String columnName) dropColumns
(String... columnNames) Creates a new table without thecolumnNames
fromthis
.exactJoin
(Table rightTable, Collection<? extends JoinMatch> columnsToMatch, Collection<? extends JoinAddition> columnsToAdd) Perform an exact-join with therightTable
.flatten()
Creates a version of this table with a flat RowSet.floatColumnIterator
(@NotNull String columnName) protected final @Nullable Table
<T> ColumnSource<T>
getColumnSource
(String sourceName) Retrieves aColumnSource
.Map<String,
? extends ColumnSource<?>> Collection<? extends ColumnSource<?>>
getSubTable
(TrackingRowSet rowSet) Get aTable
that contains a sub-set of the rows fromthis
.head
(long size) headPct
(double percent) Provides a head that selects a dynamic number of rows based on a percent.integerColumnIterator
(@NotNull String columnName) boolean
isFlat()
Return true if this table is guaranteed to be flat.join
(Table rightTable, Collection<? extends JoinMatch> columnsToMatch, Collection<? extends JoinAddition> columnsToAdd, int reserveBits) Perform a cross join with therightTable
.lazyUpdate
(Collection<? extends Selectable> newColumns) Compute column formulas on demand.longColumnIterator
(@NotNull String columnName) moveColumns
(int index, String... columnsToMove) Produce a new table with the specified columns moved to the specifiedindex
.naturalJoin
(Table rightTable, Collection<? extends JoinMatch> columnsToMatch, Collection<? extends JoinAddition> columnsToAdd) Perform an exact-join with therightTable
.<DATA_TYPE>
CloseableIterator<DATA_TYPE>objectColumnIterator
(@NotNull String columnName) partitionBy
(boolean dropKeys, String... keyColumnNames) Create aPartitionedTable
from this table, partitioned according to the specified key columns.partitionedAggBy
(Collection<? extends Aggregation> aggregations, boolean preserveEmpty, Table initialGroups, String... keyColumnNames) Convenience method that performs anTableOperations.aggBy(io.deephaven.api.agg.Aggregation)
and wraps the result in aPartitionedTable
.rangeJoin
(@NotNull Table rightTable, @NotNull Collection<? extends JoinMatch> exactMatches, @NotNull RangeJoinMatch rangeMatch, @NotNull Collection<? extends Aggregation> aggregations) Perform a range join withrightTable
.void
removeUpdateListener
(ShiftObliviousListener listener) Unsubscribe the supplied listener.void
removeUpdateListener
(TableUpdateListener listener) Unsubscribe the supplied listener.protected final void
removeUpdateListenerUncoalesced
(@NotNull TableUpdateListener listener) renameColumns
(Collection<Pair> pairs) Produce a new table with the specified columns renamed using the specifiedpairs
.reverse()
rollup
(Collection<? extends Aggregation> aggregations, boolean includeConstituents, Collection<? extends ColumnName> groupByColumns) Create a rollup table.select
(Collection<? extends Selectable> columns) selectDistinct
(Collection<? extends Selectable> columns) protected final void
setCoalesced
(Table coalesced) Proactively set the coalesced result table.shortColumnIterator
(@NotNull String columnName) long
size()
The size of this data structure.long
slice
(long firstPositionInclusive, long lastPositionExclusive) Extracts a subset of a table by row position.slicePct
(double startPercentInclusive, double endPercentExclusive) Extracts a subset of a table by row percentages.snapshot()
Creates a table with a single static snapshot ofthis
.snapshotWhen
(Table trigger, SnapshotWhenOptions options) Creates a table that captures a snapshot ofthis
whenevertrigger
updates.sort
(Collection<SortColumn> columnsToSortBy) tail
(long size) tailPct
(double percent) Provides a tail that selects a dynamic number of rows based on a percent.Create a hierarchical tree table.ungroup
(boolean nullFill, Collection<? extends ColumnName> columnsToUngroup) Ungroups a table by expanding columns of arrays or vectors into columns of singular values, creating one row in the output table for each value in the columns to be ungrouped.update
(Collection<? extends Selectable> columns) updateBy
(@NotNull UpdateByControl control, @NotNull Collection<? extends UpdateByOperation> ops, @NotNull Collection<? extends ColumnName> byColumns) Creates a table with additional columns calculated from window-based aggregations of columns in its parent.updateView
(Collection<? extends Selectable> columns) view
(Collection<? extends Selectable> columns) whereIn
(Table rightTable, Collection<? extends JoinMatch> columnsToMatch) Filtersthis
table based on the set of values in therightTable
.whereNotIn
(Table rightTable, Collection<? extends JoinMatch> columnsToMatch) Filtersthis
table based on the set of values not in therightTable
.wouldMatch
(WouldMatchPair... matchers) A table operation that applies the supplied predicate to each row in the table and produces columns containing the pass/fail result of the predicate application.Methods inherited from class io.deephaven.engine.table.impl.BaseTable
addParentReference, append, checkAvailableColumns, copyAttributes, copyAttributes, copySortableColumns, createSnapshotControlIfRefreshing, destroy, getDefinition, getDescription, getLastNotificationStep, getNotificationQueue, getUpdateGraph, hasListeners, initializeWithSnapshot, isAddOnly, isAppendOnly, isBlink, isFailed, isRefreshing, isSystemicObject, markSystemic, notifyListeners, notifyListeners, notifyListenersOnError, removeBlink, satisfied, setLastNotificationStep, setRefreshing, setTotalsTable, shouldCopyAttribute, toString, withKeys, withUniqueKeys
Methods inherited from class io.deephaven.engine.table.impl.BaseGridAttributes
clearSortingRestrictions, getSortableColumns, restrictSortTo, setColumnDescriptions, setLayoutHints, setSortableColumns, withColumnDescription, withColumnDescriptions, withDescription
Methods inherited from class io.deephaven.engine.table.impl.LiveAttributeMap
copy, copyAttributes, getAttribute, getAttributeKeys, getAttributes, getAttributes, hasAttribute, prepareReturnCopy, prepareReturnThis, published, retainingAttributes, setAttribute, setAttribute, withAttributes, withAttributes, withoutAttributes
Methods inherited from class io.deephaven.engine.liveness.LivenessArtifact
manageWithCurrentScope
Methods inherited from class io.deephaven.engine.liveness.ReferenceCountedLivenessNode
getWeakReference, initializeTransientFieldsForLiveness, onReferenceCountAtZero, tryManage, tryUnmanage, tryUnmanage
Methods inherited from class io.deephaven.engine.liveness.ReferenceCountedLivenessReferent
dropReference, tryRetainReference
Methods inherited from class io.deephaven.util.referencecounting.ReferenceCounted
decrementReferenceCount, forceReferenceCountToZero, getReferenceCountDebug, incrementReferenceCount, resetReferenceCount, tryDecrementReferenceCount, tryIncrementReferenceCount
Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait
Methods inherited from interface io.deephaven.engine.table.AttributeMap
getAttribute, getAttributeKeys, getAttributes, getAttributes, hasAttribute, retainingAttributes, withAttributes, withAttributes, withoutAttributes
Methods inherited from interface io.deephaven.engine.table.GridAttributes
clearSortingRestrictions, restrictSortTo, setLayoutHints, withColumnDescription, withColumnDescriptions, withDescription
Methods inherited from interface io.deephaven.engine.liveness.LivenessManager
manage, tryManage
Methods inherited from interface io.deephaven.engine.liveness.LivenessNode
tryUnmanage, tryUnmanage, unmanage, unmanage
Methods inherited from interface io.deephaven.engine.liveness.LivenessReferent
dropReference, getReferentDescription, getWeakReference, retainReference, tryRetainReference
Methods inherited from interface io.deephaven.util.datastructures.LongSizedDataStructure
intSize, intSize
Methods inherited from interface io.deephaven.engine.updategraph.NotificationQueue.Dependency
getUpdateGraph
Methods inherited from interface io.deephaven.engine.table.impl.NotificationStepReceiver
initializeLastNotificationStep
Methods inherited from interface io.deephaven.engine.table.impl.TableDefaults
addUpdateListener, applyToAllBy, applyToAllBy, applyToAllBy, close, dropColumnFormats, formatColumns, formatColumnWhere, formatRowWhere, getColumnSource, getColumnSource, hasColumns, hasColumns, headBy, isEmpty, join, meta, moveColumnsDown, moveColumnsUp, numColumns, partitionBy, releaseCachedResources, renameAllColumns, renameColumns, rollup, rollup, rollup, rollup, rollup, selectDistinct, snapshotWhen, snapshotWhen, tailBy, wouldMatch
Methods inherited from interface io.deephaven.api.TableOperationsDefaults
absSumBy, absSumBy, absSumBy, absSumBy, aggAllBy, aggAllBy, aggAllBy, aggBy, aggBy, aggBy, aggBy, aggBy, aggBy, aggBy, aj, aj, avgBy, avgBy, avgBy, avgBy, countBy, countBy, countBy, countBy, dropColumns, dropColumns, exactJoin, exactJoin, firstBy, firstBy, firstBy, firstBy, groupBy, groupBy, groupBy, join, join, join, lastBy, lastBy, lastBy, lastBy, lazyUpdate, maxBy, maxBy, maxBy, maxBy, medianBy, medianBy, medianBy, medianBy, minBy, minBy, minBy, minBy, naturalJoin, naturalJoin, raj, raj, rangeJoin, select, select, selectDistinct, sort, sortDescending, stdBy, stdBy, stdBy, stdBy, sumBy, sumBy, sumBy, sumBy, ungroup, ungroup, ungroup, ungroup, update, updateBy, updateBy, updateBy, updateBy, updateBy, updateBy, updateBy, updateBy, updateView, varBy, varBy, varBy, varBy, view, wavgBy, wavgBy, wavgBy, wavgBy, where, whereIn, whereNotIn, wsumBy, wsumBy, wsumBy, wsumBy
-
Constructor Details
-
UncoalescedTable
public UncoalescedTable(@NotNull @NotNull TableDefinition definition, @NotNull @NotNull String description)
-
-
Method Details
-
doCoalesce
Produce the actual coalesced result table, suitable for caching.Note that if this table must have listeners registered, etc, setting these up is the implementation's responsibility.
Also note that the implementation should copy attributes, as in
copyAttributes(resultTable, CopyAttributeOperation.Coalesce)
.- Returns:
- The coalesced result table, suitable for caching
-
coalesce
Description copied from interface:Table
Explicitly ensure that any work needed to make a table addressable, iterable, or queryable has been done, and return the coalesced child table if appropriate.- Returns:
- This table, or a fully-coalesced child
-
setCoalesced
Proactively set the coalesced result table. SeedoCoalesce()
for the caller's responsibilities. Note that it is an error to call this more than once with a non-null input.- Parameters:
coalesced
- The coalesced result table, suitable for caching
-
getCoalesced
-
addUpdateListenerUncoalesced
-
addUpdateListenerUncoalesced
protected final boolean addUpdateListenerUncoalesced(@NotNull @NotNull TableUpdateListener listener, long requiredLastNotificationStep) -
removeUpdateListenerUncoalesced
protected final void removeUpdateListenerUncoalesced(@NotNull @NotNull TableUpdateListener listener) -
sizeForInstrumentation
public long sizeForInstrumentation() -
isFlat
public boolean isFlat()Description copied from interface:Table
Return true if this table is guaranteed to be flat. The RowSet of a flat table will be from 0...numRows-1. -
size
public long size()Description copied from interface:LongSizedDataStructure
The size of this data structure.- Returns:
- The size
-
getRowSet
- Returns:
- The
TrackingRowSet
that exposes the row keys present in this Table
-
getColumnSource
Description copied from interface:Table
Retrieves aColumnSource
. It is conveniently cast toColumnSource<Object>
using the type that caller expects. This differs fromTable.getColumnSource(String, Class)
which uses the providedClass
object to verify that the data type is a subclass of the expected class.The success of this call is equivalent to
getDefinition().checkColumn(sourceName)
, which is the preferred way to check for compatibility in scenarios where the caller does not want the implementation to potentially invokeTable.coalesce()
.- Type Parameters:
T
- The target type, as a type parameter. Inferred from context.- Parameters:
sourceName
- The name of the column- Returns:
- The column source for
sourceName
, parameterized byT
- See Also:
-
getColumnSourceMap
-
getColumnSources
-
columnIterator
-
characterColumnIterator
public CloseablePrimitiveIteratorOfChar characterColumnIterator(@NotNull @NotNull String columnName) -
byteColumnIterator
-
shortColumnIterator
-
integerColumnIterator
-
longColumnIterator
-
floatColumnIterator
-
doubleColumnIterator
-
objectColumnIterator
public <DATA_TYPE> CloseableIterator<DATA_TYPE> objectColumnIterator(@NotNull @NotNull String columnName) -
where
-
wouldMatch
Description copied from interface:Table
A table operation that applies the supplied predicate to each row in the table and produces columns containing the pass/fail result of the predicate application. This is similar toTableOperations.where(String...)
except that instead of selecting only rows that meet the criteria, new columns are added with the result of the comparison.- Returns:
- a table with new columns containing the filter result for each row.
-
whereIn
Description copied from interface:TableOperations
Filtersthis
table based on the set of values in therightTable
.Note that when the
rightTable
ticks, all of the rows inthis
table are going to be re-evaluated, thus the intention is that therightTable
is fairly slow moving compared withthis
table.- Parameters:
rightTable
- the filtering table.columnsToMatch
- the columns to match between the two tables- Returns:
- a new table filtered on right table
-
whereNotIn
Description copied from interface:TableOperations
Filtersthis
table based on the set of values not in therightTable
.Note that when the
rightTable
ticks, all of the rows inthis
table are going to be re-evaluated, thus the intention is that therightTable
is fairly slow moving compared withthis
table.- Parameters:
rightTable
- the filtering table.columnsToMatch
- the columns to match between the two tables- Returns:
- a new table filtered on right table
-
select
-
selectDistinct
-
update
-
lazyUpdate
Description copied from interface:TableOperations
Compute column formulas on demand.Lazy update defers computation until required for a set of values, and caches the results for a set of input values. This uses less RAM than an update statement when you have a smaller set of unique values. Less computation than an updateView is needed, because the results are saved in a cache.
If you have many unique values, you should instead use an update statement, which will have more memory efficient structures. Values are never removed from the lazyUpdate cache, so it should be used judiciously on a ticking table.
- Parameters:
newColumns
- the columns to add- Returns:
- a new Table with the columns added; to be computed on demand
-
view
-
updateView
-
dropColumns
Description copied from interface:TableOperations
Creates a new table without thecolumnNames
fromthis
.- Parameters:
columnNames
- the columns to drop- Returns:
- the table
-
renameColumns
Description copied from interface:Table
Produce a new table with the specified columns renamed using the specifiedpairs
. The renames are simultaneous and unordered, enabling direct swaps between column names. The resulting table retains the original column ordering after applying the specified renames.IllegalArgumentException
will be thrown:- if a source column does not exist
- if a source column is used more than once
- if a destination column is used more than once
- Parameters:
pairs
- The columns to rename- Returns:
- The new table, with the columns renamed
-
moveColumns
Description copied from interface:Table
Produce a new table with the specified columns moved to the specifiedindex
. Column indices begin at 0. Columns can be renamed with the usual syntax, i.e."NewColumnName=OldColumnName")
. The renames are simultaneous and unordered, enabling direct swaps between column names. The resulting table retains the original column ordering except for the specified columns, which are inserted at the specified index, in the order ofcolumnsToMove
, after the effects of applying any renames.IllegalArgumentException
will be thrown:- if a source column does not exist
- if a source column is used more than once
- if a destination column is used more than once
Values of
index
outside the range of 0 to the number of columns in the table (exclusive) will be clamped to the nearest valid index.- Parameters:
index
- The index to which the specified columns should be movedcolumnsToMove
- The columns to move to the specified index (and, optionally, to rename)- Returns:
- The new table, with the columns rearranged as explained above
-
head
-
tail
-
slice
Description copied from interface:Table
Extracts a subset of a table by row position.If both firstPosition and lastPosition are positive, then the rows are counted from the beginning of the table. The firstPosition is inclusive, and the lastPosition is exclusive. The
TableOperations.head(long)
(N) call is equivalent to slice(0, N). The firstPosition must be less than or equal to the lastPosition.If firstPosition is positive and lastPosition is negative, then the firstRow is counted from the beginning of the table, inclusively. The lastPosition is counted from the end of the table. For example, slice(1, -1) includes all rows but the first and last. If the lastPosition would be before the firstRow, the result is an emptyTable.
If firstPosition is negative, and lastPosition is zero, then the firstRow is counted from the end of the table, and the end of the slice is the size of the table. slice(-N, 0) is equivalent to
TableOperations.tail(long)
(N).If the firstPosition is negative and the lastPosition is negative, they are both counted from the end of the table. For example, slice(-2, -1) returns the second to last row of the table.
If firstPosition is negative and lastPosition is positive, then firstPosition is counted from the end of the table, inclusively. The lastPosition is counted from the beginning of the table, exclusively. For example, slice(-3, 5) returns all rows starting from the third-last row to the fifth row of the table. If there are no rows between these positions, the function will return an empty table.
- Parameters:
firstPositionInclusive
- the first position to include in the resultlastPositionExclusive
- the last position to include in the result- Returns:
- a new Table, which is the request subset of rows from the original table
-
slicePct
Description copied from interface:Table
Extracts a subset of a table by row percentages.Returns a subset of table in the range [floor(startPercentInclusive * sizeOfTable), floor(endPercentExclusive * sizeOfTable)). For example, for a table of size 10, slicePct(0.1, 0.7) will return a subset from the second row to the seventh row. Similarly, slicePct(0, 1) would return the entire table (because row positions run from 0 to size-1). The percentage arguments must be in range [0,1], otherwise the function returns an error.
- Parameters:
startPercentInclusive
- the starting percentage point for rows to include in the result, range [0, 1]endPercentExclusive
- the ending percentage point for rows to include in the result, range [0, 1]- Returns:
- a new Table, which is the requested subset of rows from the original table
-
headPct
Description copied from interface:Table
Provides a head that selects a dynamic number of rows based on a percent.- Parameters:
percent
- the fraction of the table to return between [0, 1]. The number of rows will be rounded up. For example if there are 3 rows, headPct(50) returns the first two rows. For percent values outside [0, 1], the function will throw an exception.
-
tailPct
Description copied from interface:Table
Provides a tail that selects a dynamic number of rows based on a percent.- Parameters:
percent
- the fraction of the table to return between [0, 1]. The number of rows will be rounded up. For example if there are 3 rows, tailPct(50) returns the last two rows. For percent values outside [0, 1], the function will throw an exception.
-
exactJoin
public Table exactJoin(Table rightTable, Collection<? extends JoinMatch> columnsToMatch, Collection<? extends JoinAddition> columnsToAdd) Description copied from interface:TableOperations
Perform an exact-join with therightTable
.Similar to
TableOperations.naturalJoin(Object, Collection, Collection)
, but requires that exactly one match from therightTable
.- Parameters:
rightTable
- The right side table on the join.columnsToMatch
- The match pair conditions.columnsToAdd
- The columns from the right side that need to be added to the left side as a result of the match.- Returns:
- the exact-joined table
-
asOfJoin
public Table asOfJoin(Table rightTable, Collection<? extends JoinMatch> exactMatches, AsOfJoinMatch asOfMatch, Collection<? extends JoinAddition> columnsToAdd) -
naturalJoin
public Table naturalJoin(Table rightTable, Collection<? extends JoinMatch> columnsToMatch, Collection<? extends JoinAddition> columnsToAdd) Description copied from interface:TableOperations
Perform an exact-join with therightTable
.Requires zero or one match from the
rightTable
.- Parameters:
rightTable
- The right side table on the join.columnsToMatch
- The match pair conditions.columnsToAdd
- The columns from the right side that need to be added to the left side as a result of the match.- Returns:
- the natural-joined table
-
join
public Table join(Table rightTable, Collection<? extends JoinMatch> columnsToMatch, Collection<? extends JoinAddition> columnsToAdd, int reserveBits) Description copied from interface:TableOperations
Perform a cross join with therightTable
.Returns a table that is the cartesian product of left rows X right rows, with one column for each of
this
table's columns, and one column corresponding to each of therightTable
's columns that are included in thecolumnsToAdd
argument. The rows are ordered first by thethis
table then by therightTable
. IfcolumnsToMatch
is non-empty then the product is filtered by the supplied match conditions.To efficiently produce updates, the bits that represent a key for a given row are split into two. Unless specified, join reserves 16 bits to represent a right row. When there are too few bits to represent all of the right rows for a given aggregation group the table will shift a bit from the left side to the right side. The default of 16 bits was carefully chosen because it results in an efficient implementation to process live updates.
An io.deephaven.engine.table.impl.util.OutOfKeySpaceException is thrown when the total number of bits needed to express the result table exceeds that needed to represent Long.MAX_VALUE. There are a few work arounds:
- If the left table is sparse, consider flattening the left table.
- If there are no key-columns and the right table is sparse, consider flattening the right table.
- If the maximum size of a right table's group is small, you can reserve fewer bits by setting
reserveBits
on initialization.Note: If you know that a given group has at most one right-row then you should prefer using
TableOperations.naturalJoin(Object, Collection, Collection)
.- Parameters:
rightTable
- The right side table on the join.columnsToMatch
- The match pair conditions.columnsToAdd
- The columns from the right side that need to be added to the left side as a result of the match.reserveBits
- The number of bits to reserve for rightTable groups.- Returns:
- a new table joined according to the specification in columnsToMatch and columnsToAdd
-
rangeJoin
public Table rangeJoin(@NotNull @NotNull Table rightTable, @NotNull @NotNull Collection<? extends JoinMatch> exactMatches, @NotNull @NotNull RangeJoinMatch rangeMatch, @NotNull @NotNull Collection<? extends Aggregation> aggregations) Description copied from interface:TableOperations
Perform a range join withrightTable
. For each row inthis
Table, this operation joinsaggregations
over a range of responsive rows fromrightTable
according to zero-or-more exact join matches and one range join match.Matching Rules
The exact join matches identify possibly-responsive rows according to exactly matching values between the left exact match columns and the right exact match columns, similar to other join operations. The range join match bounds the beginning of the responsive range for a given output row by the relationship between a left start column and a right range column, governed by the rangeStartRule, and bounds the end of the responsive range for a given output row by the relationship between a left end column and the same right range column, governed by the rangeEndRule.
Right Table Row-Inclusion Criteria and Relative Ordering Requirements
Rows from
rightTable
withnull
orNaN
values for the right range column are discarded; that is, they are never included in the responsive range for any output row. Within a group sharing the same values for the right exact match columns,rightTable
must be relatively ordered (as ifsorted
) according to the right range column for all rows that are not discarded.Special Cases
In order to produce aggregated output, it is required that the two relative match expressions define a range of values to determine the responsive rows to aggregate. There are a few noteworthy special cases of ranges.
- empty range
- An empty range occurs for any left row with no responsive right rows. That is, no non-
null
, non-NaN
right rows were found using the exact join matches, or none were in range according to the range join match. - single-value ranges
- A single-value range is a range where the left row’s values for the left start column and left end
column are equal and both relative matches are inclusive (
<=
and>=
, respectively). For a single-value range, only rows within the bucket where the right range column matches the single value are included in the output aggregations. - invalid ranges
- An invalid range occurs in two scenarios: First, when the range is inverted, i.e. when the value of
the left start column is greater than the value of the left end column. Second, when either relative-match is
exclusive (
<
or>
) and the value in the left start column is equal to the value in the left end column (becausevalue < value == false
). Specifying "allow preceding" or "allow following" for either rule will not constitute an exception to either of these defined scenarios. For invalid ranges, the result row will benull
for all aggregation output columns. - undefined ranges
- An undefined range occurs when either the left start column or the left end column is
NaN
. For rows with an undefined range, the corresponding output values will benull
(as with invalid ranges). - Unbounded Ranges
- A partially or fully unbounded range occurs when either the left start column or the left end column
is
null
. If the left start column value isnull
the range is unbounded at the beginning, and all matched right rows will be included if they respect the match rule for the left end column. If the left end column value isnull
the range is unbounded at the end, and all matched right rows will be included if they respect the match rule for the left start column. If both the left start column and left end column values arenull
the range is unbounded in both directions, and all matched right rows will be included.
- Parameters:
rightTable
- The Table to join withexactMatches
- Possibly-empty collection ofjoin matches
that dictate exact-match criteria. That is, rows fromrightTable
that might be responsive to rows fromthis
Table will have identical values for the column pairs expressed by these matches.rangeMatch
- Specifies the range match criteria for determining the responsive rows fromrightTable
for each row fromthis
Table, within the buckets created by matching on theexactMatches
aggregations
- Theaggregations
to perform over the responsive ranges fromrightTable
for each row fromthis
Table- Returns:
- The result Table
-
aggAllBy
-
aggBy
@ConcurrentMethod public Table aggBy(Collection<? extends Aggregation> aggregations, boolean preserveEmpty, Table initialGroups, Collection<? extends ColumnName> groupByColumns) Description copied from interface:TableOperations
Produce an aggregated result by groupingthis
according to thegroupByColumns
and applyingaggregations
to each resulting group of rows. The result table will have one row per group, ordered by the encounter order withinthis
, thereby ensuring that the row key for a given group never changes.- Parameters:
aggregations
- Theaggregations
to applypreserveEmpty
- Whether to keep result rows for groups that are initially empty or become empty as a result of updates. Each aggregation operator defines its own value for empty groups.initialGroups
- A table whose distinct combinations of values for thegroupByColumns
should be used to create an initial set of aggregation groups. All other columns are ignored. This is useful in combination withpreserveEmpty == true
to ensure that particular groups appear in the result table, or withpreserveEmpty == false
to control the encounter order for a collection of groups and thus their relative order in the result. Changes toinitialGroups
are not expected or handled; ifinitialGroups
is a refreshing table, only its contents at instantiation time will be used. IfinitialGroups == null
, the result will be the same as if a table with no rows was supplied.groupByColumns
- Thecolumns
to group by- Returns:
- A new table aggregating the rows of
this
-
headBy
-
tailBy
-
ungroup
Description copied from interface:TableOperations
Ungroups a table by expanding columns of arrays or vectors into columns of singular values, creating one row in the output table for each value in the columns to be ungrouped. Columns that are not ungrouped have their values duplicated in each output row corresponding to a given input row.- Parameters:
nullFill
- indicates if the ungrouped table should allow disparate sized arrays filling shorter columns with null values. If set to false, then all arrays should be the same length.columnsToUngroup
- the columns to ungroup- Returns:
- the ungrouped table
-
partitionBy
Description copied from interface:Table
Create aPartitionedTable
from this table, partitioned according to the specified key columns.The underlying partitioned table backing the result contains each row in
this
table in exactly one of the result's constituent tables.- Parameters:
dropKeys
- Whether to drop key columns in the output constituent tableskeyColumnNames
- The names of the key columns to partition by- Returns:
- A
PartitionedTable
keyed bykeyColumnNames
-
partitionedAggBy
@ConcurrentMethod public PartitionedTable partitionedAggBy(Collection<? extends Aggregation> aggregations, boolean preserveEmpty, Table initialGroups, String... keyColumnNames) Description copied from interface:Table
Convenience method that performs anTableOperations.aggBy(io.deephaven.api.agg.Aggregation)
and wraps the result in aPartitionedTable
. Ifaggregations
does not include apartition
, one will be added automatically with the default constituent column name and behavior used inTable.partitionBy(String...)
.- Parameters:
aggregations
- Theaggregations
to applypreserveEmpty
- Whether to keep result rows for groups that are initially empty or become empty as a result of updates. Each aggregation operator defines its own value for empty groups.initialGroups
- A table whose distinct combinations of values for thegroupByColumns
should be used to create an initial set of aggregation groups. All other columns are ignored. This is useful in combination withpreserveEmpty == true
to ensure that particular groups appear in the result table, or withpreserveEmpty == false
to control the encounter order for a collection of groups and thus their relative order in the result. Changes toinitialGroups
are not expected or handled; ifinitialGroups
is a refreshing table, only its contents at instantiation time will be used. IfinitialGroups == null
, the result will be the same as if a table with no rows was supplied.keyColumnNames
- The names of the key columns to aggregate by- Returns:
- A
PartitionedTable
keyed bykeyColumnNames
-
rollup
@ConcurrentMethod public RollupTable rollup(Collection<? extends Aggregation> aggregations, boolean includeConstituents, Collection<? extends ColumnName> groupByColumns) Description copied from interface:Table
Create a rollup table.A rollup table aggregates by the specified columns, and then creates a hierarchical table which re-aggregates using one less aggregation column on each level. The column that is no longer part of the aggregation key is replaced with null on each level.
- Parameters:
aggregations
- The aggregations to performincludeConstituents
- set to true to include the constituent rows at the leaf levelgroupByColumns
- the columns to group by- Returns:
- a hierarchical table with the rollup applied
-
tree
Description copied from interface:Table
Create a hierarchical tree table.The structure of the table is encoded by an "id" and a "parent" column. The id column should represent a unique identifier for a given row, and the parent column indicates which row is the parent for a given row. Rows that have a
null
parent are part of the "root" table.It is possible for rows to be "orphaned" if their parent is non-
null
and does not exist in the table. SeeTreeTable.promoteOrphans(Table, String, String)
.- Parameters:
idColumn
- The name of a column containing a unique identifier for a particular row in the tableparentColumn
- The name of a column containing the parent's identifier,null
for rows that are part of the root table- Returns:
- A
TreeTable
organized according to the parent-child relationships expressed byidColumn
andparentColumn
-
updateBy
public Table updateBy(@NotNull @NotNull UpdateByControl control, @NotNull @NotNull Collection<? extends UpdateByOperation> ops, @NotNull @NotNull Collection<? extends ColumnName> byColumns) Description copied from interface:TableOperations
Creates a table with additional columns calculated from window-based aggregations of columns in its parent. The aggregations are defined by theoperations
, which support incremental aggregation over the corresponding rows in the parent table. The aggregations will apply position or time-based windowing and compute the results for the row group (as determined by thebyColumns
).- Parameters:
control
- thecontrol
to use when updating the table.ops
- the operations to apply to the table.byColumns
- the columns to group by before applying.- Returns:
- a table with the same rowSet, with the specified operations applied to each group defined by the
byColumns
-
sort
-
reverse
-
snapshot
Description copied from interface:TableOperations
Creates a table with a single static snapshot ofthis
.- Returns:
- the snapshot
-
snapshotWhen
Description copied from interface:TableOperations
Creates a table that captures a snapshot ofthis
whenevertrigger
updates.- Parameters:
trigger
- the trigger tableoptions
- the snapshot options- Returns:
- the snapshotting table
- See Also:
-
getSubTable
Description copied from interface:Table
Get aTable
that contains a sub-set of the rows fromthis
. The result will share the samecolumn sources
anddefinition
as this table.The result will not update on its own. The caller must also establish an appropriate listener to update
rowSet
and propagateupdates
.- Parameters:
rowSet
- Therow set
for the result- Returns:
- A new sub-table
-
apply
Description copied from interface:Table
Applies a function to this table.This is useful if you have a reference to a table or a proxy and want to run a series of operations against the table without each individual operation resulting in an RMI.
- Type Parameters:
R
- the return type of function- Parameters:
function
- the function to run, its single argument will be this table- Returns:
- the return value of function
-
flatten
Description copied from interface:Table
Creates a version of this table with a flat RowSet. -
awaitUpdate
Description copied from interface:Table
Wait for updates to this Table. Should not be invoked from a
TableListener
or othernotification
on this Table'supdate graph
. It may be suitable to wait from another update graph if doing so does not introduce any cycles.In some implementations, this call may also terminate in case of interrupt or spurious wakeup.
- Specified by:
awaitUpdate
in interfaceTable
- Overrides:
awaitUpdate
in classBaseTable<IMPL_TYPE extends UncoalescedTable<IMPL_TYPE>>
- Throws:
InterruptedException
- In the event this thread is interrupted- See Also:
-
awaitUpdate
Description copied from interface:Table
Wait for updates to this Table. Should not be invoked from a
TableListener
or othernotification
on this Table'supdate graph
. It may be suitable to wait from another update graph if doing so does not introduce any cycles.In some implementations, this call may also terminate in case of interrupt or spurious wakeup.
- Specified by:
awaitUpdate
in interfaceTable
- Overrides:
awaitUpdate
in classBaseTable<IMPL_TYPE extends UncoalescedTable<IMPL_TYPE>>
- Parameters:
timeout
- The maximum time to wait in milliseconds.- Returns:
- false if the timeout elapses without notification, true otherwise.
- Throws:
InterruptedException
- In the event this thread is interrupted- See Also:
-
addUpdateListener
Description copied from interface:Table
Subscribe for updates to this table. After the optional initial image,listener
will be invoked via theNotificationQueue
associated with this Table.- Specified by:
addUpdateListener
in interfaceTable
- Overrides:
addUpdateListener
in classBaseTable<IMPL_TYPE extends UncoalescedTable<IMPL_TYPE>>
- Parameters:
listener
- listener for updatesreplayInitialImage
- true to process updates for all initial rows in the table plus all changes; false to only process changes
-
addUpdateListener
Description copied from interface:Table
Subscribe for updates to this table.listener
will be invoked via theNotificationQueue
associated with this Table.- Specified by:
addUpdateListener
in interfaceTable
- Overrides:
addUpdateListener
in classBaseTable<IMPL_TYPE extends UncoalescedTable<IMPL_TYPE>>
- Parameters:
listener
- listener for updates
-
addUpdateListener
public boolean addUpdateListener(@NotNull @NotNull TableUpdateListener listener, long requiredLastNotificationStep) Description copied from interface:Table
Subscribe for updates to this table if its last notification step matchesrequiredLastNotificationStep
.listener
will be invoked via theNotificationQueue
associated with this Table.- Specified by:
addUpdateListener
in interfaceTable
- Overrides:
addUpdateListener
in classBaseTable<IMPL_TYPE extends UncoalescedTable<IMPL_TYPE>>
- Parameters:
listener
- listener for updatesrequiredLastNotificationStep
- the expected last notification step to match- Returns:
- true if the listener was added, false if the last notification step requirement was not met
-
removeUpdateListener
Description copied from interface:Table
Unsubscribe the supplied listener.- Specified by:
removeUpdateListener
in interfaceTable
- Overrides:
removeUpdateListener
in classBaseTable<IMPL_TYPE extends UncoalescedTable<IMPL_TYPE>>
- Parameters:
listener
- listener for updates
-
removeUpdateListener
Description copied from interface:Table
Unsubscribe the supplied listener.- Specified by:
removeUpdateListener
in interfaceTable
- Overrides:
removeUpdateListener
in classBaseTable<IMPL_TYPE extends UncoalescedTable<IMPL_TYPE>>
- Parameters:
listener
- listener for updates
-