Arrays

This guide shows you how to work with arrays in query strings.

Arrays are an invaluable tool for grouping related data. They provide an easy way to access previous and future values in time series datasets. Support for arrays is built into the Deephaven Query Language (DQL).

Array column types

Array columns fall into one of three categories of data type.

Array columns

Array columns are Java arrays of primitive types. For example, the following query creates a table with a single row containing an array of primitive integers.

source = emptyTable(1).update("X = new int[]{1, 2, 3}")
sourceMeta = source.meta()

You can also use Groovy closures to create Java primitive array columns:

listFunc = { -> [4, 5, 6] }

source = emptyTable(1).update("ArrayFromGroovy = listFunc()")
sourceMeta = source.meta()

The Deephaven engine can seamlessly work with these column types.

Vector columns

source = emptyTable(5).update("X = ii % 2", "Y = ii").groupBy("X")
sourceMeta = source.meta()

The Deephaven engine can seamlessly work with these column types.

Vector columns

Vector columns arise from common table operations including grouping. These vector columns are used in dedicated aggregations, combined aggregations, updateBy, and more. The following example creates a vector column by calling groupBy:

source = emptyTable(5).update("X = ii % 2", "Y = ii").groupBy("X")
sourceMeta = source.meta()

Convert between arrays and vectors

Since Deephaven tables commonly use both Java primitive arrays and Deephaven vectors, it's useful to convert between the two. The following example converts between both vector and array columns:

source = emptyTable(10).update("Vector = ii").groupBy()
result = source.update(
  "ArrayFromVector = array(Vector)",
  "VectorFromArray = vec(ArrayFromVector)",
)
resultMeta = result.meta()

Create array columns

By grouping

Arrays can be created using the groupBy method to group data.

source = emptyTable(10).update("X = (ii % 2 == 0) ? `A` : `B` ", "Y = ii")

result = source.groupBy("X")

With aggregations

Certain aggregations create array columns. For example, the following operations create array columns:

The following example calls RollingGroup to create an array column:

source = emptyTable(10).update(
  "Key = (ii % 2 == 0) ? `A` : `B`", "Value = randomDouble(0, 1)"
)

result = source.updateBy(RollingGroup(3, "TickGroup=Value"), "Key")
resultMeta = result.meta()

Using the underscore operator

Every column in a table has an associated array variable, which can be accessed using the underscore (_) operator. This operator is specific to Deephaven. For example, a column called X can be accessed as an array by using the column name X_:

source = emptyTable(10).update("X = ii")
result = source.update("A = X_")

Get array length

The len method returns the length of the given input. This is useful in query strings where you need to get the size of a Vector or a Java array.

source = emptyTable(10).update("X = i").groupBy()
result = source.update("LenX = len(X)")

Access array elements

The square bracket operators [] are used to access elements in array columns. The following example uses these operators to access the previous and next elements in the array, as well as print the size of the array:

source = emptyTable(10).update("X = ii")
result = source.update("A = X_[ii - 1]", "B = X_[ii + 1]", "C = X_.size()")

Note

The first row of column A is null because there is no previous element at the zeroth array index. The last row of column B is null because there is no next element at the last array index.

Additionally, you can access specific array elements directly using indexes.

result = (
    emptyTable(10)
    .update("X = (ii % 2 == 0) ? `A` : `B` ", "Y = ii")
    .groupBy("X")
)
indexingResult = result.update("Element2 = Y[2]", "Element3 = Y[3]")

Slice arrays

You can slice arrays into subarrays with subVector. The first input is the index at which the slice starts, while the second is the index at which the slice ends. They are inclusive and exclusive, respectively.

The following example slices the array column, then grabs specific elements from the subarray.

source = (
  emptyTable(10)
  .update("X = (ii % 2 == 0) ? `A` : `B` ", "Y = ii")
  .groupBy("X")
)
result = source.update(
  "SubArray = Y.subVector(2, 4)",
  "SubSlice = SubArray[1]",
)
resultMeta = result.meta()

Functions with array arguments

Built-in query language functions

Caution

Dedicated aggregations, Combined aggregations, and updateBy are always more performant than groupBy followed by manual calculations when used on ticking tables.

Many built-in query language methods take arrays as input. The following example uses the sum and avg functions on a column containing arrays.

result = (
  emptyTable(10)
  .update("X = (ii % 2 == 0) ? `A` : `B` ", "Y = ii")
  .groupBy("X")
)
sumResult = result.update("ArraySum = sum(Y)", "ArrayAvg = avg(Y)")

Groovy closures

Groovy closures that take arrays as input are also supported in query strings. The following example calls a Groovy closure that takes a Java array as input:

arraySum = { double[] arr ->
    double total = 0.0
    for (double value : arr) {
        total += value
    }
    return total
}

source = emptyTable(1).update(
    "ArrayColumn = new double[]{3.14, 1.23, -0.919}",
    "CallGroovy = (double)arraySum(ArrayColumn)"
)
sourceMeta = source.meta()