Python variables in query strings

The ability to use your own custom Python variables, functions, and classes in Deephaven query strings is one of its most powerful features. The use of Python variables in query strings follows some basic rules, which are outlined in this guide.

Python variables in query strings

There are three types of Python variables supported in the Deephaven Query Language (DQL). The use of variables in queries follows query scope rules similar to Python's LEGB ordering. Additionally, since the Deephaven engine is built on Java, the Python-Java boundary is an important consideration when using Python variables in query strings.

Scalars

Scalars are single values, such as numbers or booleans. They can be used directly in query strings without special syntax. The following example shows the basic use of Python variables in query strings to create new columns in a table:

from deephaven import empty_table

a = 4
b = 3.14
c = -1.91e7

source = empty_table(1).update(["A = a", "B = b", "C = c + 1"])
source_meta = source.meta_table

Strings

Like scalars, Python strings can be used in query strings with no special syntax. The following example creates a table with two string columns:

from deephaven import empty_table

my_first_string = "Hello, world!"
my_second_string = "Coding is fun."

source = empty_table(1).update(
    ["FirstString = my_first_string", "SecondString = my_second_string"]
)
source_meta = source.meta_table

Sequences

In Python, sequences include lists, NumPy arrays, and more. These can be used in query strings as well.

from deephaven import empty_table

my_list = [1, 2, 3]

source_list = empty_table(1).update("ListColumn = my_list")
source_list_meta = source_list.meta_table

Extracting Python sequence elements in the query language is also supported. The Deephaven engine is not able to infer data types from extracted list elements, so be sure to use type casts to ensure the correct resultant column types.

from deephaven import empty_table

my_list = [1, 2, 3]

source = empty_table(1).update(
    [
        "FirstElement = (int)my_list[0]",
        "SecondElement = (int)my_list[1]",
        "ThirdElement = (int)my_list[2]",
    ]
)
source_meta = source.meta_table

It's common to use jpy to convert Python sequence variables to Java primitive arrays before using them in query strings. This is often more efficient than using the Python sequence directly.

from deephaven import empty_table
import jpy

my_list = [1, 2, 3]
j_my_list = jpy.array("int", my_list)

source_java_array = empty_table(1).update("ArrayColumn = j_my_list")
source_java_array_meta = source_java_array.meta_table

Python variables in query strings