Arrays vs vectors
Deephaven tables can store multi-element data in two forms: Java arrays and Deephaven vectors. Both hold ordered collections of elements, but they differ in ways that affect how you work with them.
You create Java arrays yourself in formulas. Deephaven produces vectors when it groups rows or when a formula reads a whole column through a column array. This page compares the two, shows how to convert between them, and explains when to convert a vector to an array.
Quick comparison
| Feature | Java arrays | Deephaven vectors |
|---|---|---|
| Type examples | int[], double[], String[] | IntVector, DoubleVector, ObjectVector |
| Index type | int (max ~2.1 billion elements) | long (supports larger datasets) |
| Out-of-bounds access | Throws an ArrayIndexOutOfBoundsException | Returns the null value for its type (e.g., NULL_INT) |
| Created by | Explicit construction, array, or arrayObj | Table operations that group rows, column arrays such as X_, vec, or vecObj |
| Memory model | Owns contiguous memory | May be a view into table data |
| Mutability | Java code can reassign elements (a[i] = v) | Read-only; no methods to set elements |
| Slicing | Arrays.copyOfRange | subVector (no copy) |
Java arrays
Java arrays are standard fixed-size collections. Create them explicitly using Java syntax:
The metadata shows the columns as int[], double[], and java.lang.String[]. A Java array stores its elements itself, and it differs from a vector in three ways:
- Code that receives the array, such as a Groovy closure or Java method, can change an element (
a[i] = v). A query formula can't, because formulas can't contain assignments. - Its index is an
int, so it holds at most about 2.1 billion elements. - An out-of-bounds index throws an
ArrayIndexOutOfBoundsExceptioninstead of returning null.
Deephaven vectors
A Deephaven vector is a collection defined by the Vector interface. Each primitive type except boolean has its own subinterface, such as IntVector and DoubleVector. Other types, including Boolean and String, use ObjectVector. Table operations that group data produce vector columns:
The Y column is an IntVector, not an int[].
The following operations produce vector columns:
A vector that a table operation produces can be a view into the source table's columns. It reads its values from those columns instead of copying them.
This matters for live data. When the source table ticks, each vector that groupBy produces reflects its group's rows as of the current update cycle.
Access a column as a vector
Append _ to a column name to read the whole column as a vector. Deephaven calls X_ a column array, but it is a vector that holds every value in column X. Index into it to read another row. For example, X_[ii - 1] is the value of X one row before the current row, where ii is the current row index:
The first row's PrevValue and the last row's NextValue are null because those indices fall outside the column.
Whether a formula can use a column array depends on the table:
- On static tables and blink tables, any column-array expression works.
- On other refreshing tables, a constant offset from the row index, such as
X_[ii - 1]orX_[ii + 2], works. By default, the engine rejects any other use of a column array.
See Refreshing table restrictions for details.
Key advantages of vectors
Long-based indexing: A Java array can't hold more than about 2.1 billion elements, because its index is an int. A vector uses a long index, so a group or column array with more rows than that is still a single vector.
Safe out-of-bounds access: Reading an index outside a vector returns the null value for the vector's type, such as NULL_INT for an IntVector, instead of throwing an exception. The column-array example shows this: the first row's PrevValue is null. The same index on a Java array throws an ArrayIndexOutOfBoundsException.
Efficient slicing: The subVector method returns another view over the same data, without copying it. The following example uses updateView to keep the slice a view. update would store a materialized copy of any vector view that a formula returns:
Slice holds the elements at positions 2, 3, and 4 of X, so FirstOfSlice is 2. Slice[3] is past the end of Slice, so PastEnd is null.
Convert between arrays and vectors
Use the built-in array function to convert a vector to an array, and vec to convert an array to a vector:
The metadata shows that AsArray is a long[] and BackToVector is a LongVector, the same type as the original X column.
Supported conversions
Each row pairs a primitive array type with its vector type. vec converts the array type to the vector type, and array converts the vector type back.
| Array type | Vector type |
|---|---|
byte[] | ByteVector |
char[] | CharVector |
double[] | DoubleVector |
float[] | FloatVector |
int[] | IntVector |
long[] | LongVector |
short[] | ShortVector |
For object types, use arrayObj to convert an ObjectVector to an array, and vecObj to convert an object array, such as a String[], to an ObjectVector.
When to convert to an array
Deephaven produces vectors for grouped data and column arrays such as X_, so the decision is whether to convert them. In a formula, you usually don't need to. When a formula passes a vector to a Java method that takes an array, the engine converts the vector for you.
In the following example, Arrays.stream takes a long[], and the formula passes it the LongVector column X directly:
Total is 10, the sum of the values 0 through 4 in X, with no call to array.
This conversion applies to Java methods only. A Groovy closure receives the vector itself, not an array.
The array that the engine passes can share the vector's data, so the called code must not change it.
Convert to a Java array yourself when:
- Code changes elements in place.
arrayandarrayObjreturn a new copy, so changing it doesn't touch the vector's data. See Convert between arrays and vectors.
Keep Deephaven vectors when:
- Holding more than about 2.1 billion elements in one collection.
- Reading indices that can fall outside the collection, such as
X_[ii - 1]on the first row, where you want a null instead of an exception. - Slicing data without copying.