Arrays
This guide shows you how to work with arrays in query strings.
Arrays are an invaluable tool for grouping related data. They provide an easy way to access previous and future values in time series datasets. Support for arrays is built into the Deephaven Query Language (DQL).
Array column types
Array columns fall into one of three categories of data type.
Array columns
Array columns are Java arrays of primitive types. For example, the following query creates a table with a single row containing an array of primitive integers.
source = emptyTable(1).update("X = new int[]{1, 2, 3}")
sourceMeta = source.meta()
You can also use Groovy closures to create Java primitive array columns:
listFunc = { -> [4, 5, 6] }
source = emptyTable(1).update("ArrayFromGroovy = listFunc()")
sourceMeta = source.meta()
The Deephaven engine can seamlessly work with these column types.
Vector columns
source = emptyTable(5).update("X = ii % 2", "Y = ii").groupBy("X")
sourceMeta = source.meta()
The Deephaven engine can seamlessly work with these column types.
Vector columns
Vector columns arise from common table operations including grouping. These vector columns are used in dedicated aggregations, combined aggregations, updateBy
, and more. The following example creates a vector column by calling groupBy
:
source = emptyTable(5).update("X = ii % 2", "Y = ii").groupBy("X")
sourceMeta = source.meta()
Convert between arrays and vectors
Since Deephaven tables commonly use both Java primitive arrays and Deephaven vectors, it's useful to convert between the two. The following example converts between both vector and array columns:
source = emptyTable(10).update("Vector = ii").groupBy()
result = source.update(
"ArrayFromVector = array(Vector)",
"VectorFromArray = vec(ArrayFromVector)",
)
resultMeta = result.meta()
Create array columns
By grouping
Arrays can be created using the groupBy
method to group data.
source = emptyTable(10).update("X = (ii % 2 == 0) ? `A` : `B` ", "Y = ii")
result = source.groupBy("X")
With aggregations
Certain aggregations create array columns. For example, the following operations create array columns:
The following example calls RollingGroup
to create an array column:
source = emptyTable(10).update(
"Key = (ii % 2 == 0) ? `A` : `B`", "Value = randomDouble(0, 1)"
)
result = source.updateBy(RollingGroup(3, "TickGroup=Value"), "Key")
resultMeta = result.meta()
Using the underscore operator
Every column in a table has an associated array variable, which can be accessed using the underscore (_
) operator. This operator is specific to Deephaven. For example, a column called X
can be accessed as an array by using the column name X_
:
source = emptyTable(10).update("X = ii")
result = source.update("A = X_")
Get array length
The len
method returns the length of the given input. This is useful in query strings where you need to get the size of a Vector
or a Java array.
source = emptyTable(10).update("X = i").groupBy()
result = source.update("LenX = len(X)")
Access array elements
The square bracket operators []
are used to access elements in array columns. The following example uses these operators to access the previous and next elements in the array, as well as print the size of the array:
source = emptyTable(10).update("X = ii")
result = source.update("A = X_[ii - 1]", "B = X_[ii + 1]", "C = X_.size()")
Note
The first row of column A
is null because there is no previous element at the zeroth array index. The last row of column B
is null because there is no next element at the last array index.
Additionally, you can access specific array elements directly using indexes.
result = (
emptyTable(10)
.update("X = (ii % 2 == 0) ? `A` : `B` ", "Y = ii")
.groupBy("X")
)
indexingResult = result.update("Element2 = Y[2]", "Element3 = Y[3]")
Slice arrays
You can slice arrays into subarrays with subVector
. The first input is the index at which the slice starts, while the second is the index at which the slice ends. They are inclusive and exclusive, respectively.
The following example slices the array column, then grabs specific elements from the subarray.
source = (
emptyTable(10)
.update("X = (ii % 2 == 0) ? `A` : `B` ", "Y = ii")
.groupBy("X")
)
result = source.update(
"SubArray = Y.subVector(2, 4)",
"SubSlice = SubArray[1]",
)
resultMeta = result.meta()
Functions with array arguments
Built-in query language functions
Caution
Dedicated aggregations, Combined aggregations, and updateBy
are always more performant than groupBy
followed by manual calculations when used on ticking tables.
Many built-in query language methods take arrays as input. The following example uses the sum
and avg
functions on a column containing arrays.
result = (
emptyTable(10)
.update("X = (ii % 2 == 0) ? `A` : `B` ", "Y = ii")
.groupBy("X")
)
sumResult = result.update("ArraySum = sum(Y)", "ArrayAvg = avg(Y)")
Groovy closures
Groovy closures that take arrays as input are also supported in query strings. The following example calls a Groovy closure that takes a Java array as input:
arraySum = { double[] arr ->
double total = 0.0
for (double value : arr) {
total += value
}
return total
}
source = emptyTable(1).update(
"ArrayColumn = new double[]{3.14, 1.23, -0.919}",
"CallGroovy = (double)arraySum(ArrayColumn)"
)
sourceMeta = source.meta()