Iterate over tables with ColumnVectors

This guide shows you how to iterate over table data in Groovy queries using ColumnVectors.

Note

Iterate over a column when Groovy code must handle each value, for example to log values or pass them to other code. To compute a result such as a sum or a maximum, use a table operation such as sumBy instead. The examples below compute simple results only to keep them short. See Recipes, not loops!.

The ColumnVectors class

ColumnVectors is a helper class with factory methods that create a Vector from a table column. A vector provides bulk and random access to column data by row position.

Iterating over a vector is bulk access. The iterator walks the rows in order instead of looking up each row position separately, and for larger columns it reads values in blocks. This is faster than reading each value by position.

ColumnVectors has a typed factory method for each primitive column type, an ofObject method for all other column types, and a generic of method that is declared to return Vector<?> rather than a specific vector type. Prefer the typed method that matches the column's type:

  • For a char, byte, short, int, long, float, or double column, use the matching primitive method: ofChar, ofByte, ofShort, ofInt, ofLong, ofFloat, or ofDouble.
  • For any other type, including String and Boolean, use ofObject and pass the column's class as the third argument. Deephaven checks that the column's data type can be cast to that class and throws a ClassCastException if it can't.

Note

To read individual values by row position instead, see Extract table values. A vector's iterator reads values the same way as a table's column iterator, so create a vector when you also need its other methods, such as size, get, or copyToArray, or want to pass a Vector to other code. To iterate over a column without creating a vector, call the table's columnIterator method or a typed variant such as integerColumnIterator. See Iterate over a column.

Iterate over a column vector

The examples in this guide import ColumnVectors and use the following table:

To iterate over a specific column, create a vector that matches the column's type and then call the vector's iterator method, such as IntVector.iterator. Read each value with the iterator's matching next method, such as nextInt for an IntVector, nextDouble for a DoubleVector, or next for an ObjectVector.

The following example iterates over the X column and sums its values:

The following example iterates over the Y column and finds its largest value:

The following example iterates over the Z column, a String column read with ofObject, and counts the Even values:

Each loop above runs until hasNext returns false, at which point the iterator releases its resources. If you stop iterating early, for example with break, call the iterator's close method. The following example stops at the first X value over 90 and closes the iterator in a finally block:

Refreshing tables

For a refreshing table, a vector reflects the table's data only until the engine starts its next update cycle. To use it safely:

  • Create the vector and iterate over it within the same cycle. To do this, hold the UpdateGraph's shared lock while you create and iterate over the vector, as described in Engine locking.
  • To keep the data beyond the current cycle, copy it with copyToArray or getDirect before the cycle ends.

Commands in the Deephaven console and Application Mode startup scripts don't need the shared lock, because they already run under the UpdateGraph's exclusive lock. A table listener doesn't need it to read the table it listens to, or a table it waits for as a dependency, because the engine runs the listener only after those tables have updated.

Inside a table listener, you can read the values the table had before the current update. Pass true as the final usePreviousValues argument of a ColumnVectors factory method. For example, ColumnVectors.ofInt(myTable, 'X', true) reads the previous values of column X, where myTable is the refreshing table the listener watches. A previous-values vector has these limits:

  • It is valid only while the engine is processing an update, for example during the listener's onUpdate call.
  • Its contents are undefined in the cycle in which myTable is created.

The following listener prints the previous and current values of X each time myTable updates: