QueryScope

Variables in your formula are either taken from columns or the QueryScope, which dictates what variables and methods are accessible within a query. Furthermore, using variables from the QueryScope can reduce execution time. Changing QueryScope variables does not require query strings to be recompiled, while changing literal values in query strings may require the query string to be recompiled.

Consider the following query:

t2=t1.update("X=f.call(A,b)")

X and A are columns, f is a function, and b is some variable. The "X=f.call(A,b)" string is called a "query string".

This query string gets dynamically converted into code, which is compiled to improve execution speed while having an expressive query language. It is obvious that the columns come from the table. It is less obvious where the variable b comes from. These variables come from the "query scope".

Python

If you are at the top level of a Python program, the variables are automatically added to the scope. For example, the following query will work:

from deephaven import TableTools
z=3
result2 = TableTools.emptyTable(10).update("X=i","Y=z+X")

If you are in a function, the variables are not automatically added to the binding, so this query will not work:

def y():
    b=3
    return TableTools.emptyTable(10).update("X=i","Y=b=X")
y()

The variables can be explicitly added to the binding as in this example:

from deephaven import TableTools
from deephaven import QueryScope

def f(a, b):
   return a*b
def compute(a):
   # Values must be added to the query scope to be used in query strings.
   QueryScope.addParam("a", a)
   return TableTools.emptyTable(10).update("X=i","Y=(int)f.call(a,X)")
result1 = compute(10)
# Because variable "z" is in the top level scope of the Python file,
# it is automatically added to the scope
z=3
result2 = TableTools.emptyTable(10).update("X=i","Y=(int)f.call(z,X)")

Note

The QueryScope is the scope for the query; it does not map to the scope of the programming language. As a result, if you add an item within a code block, the item is still in the query scope after the code block returns (in other words, the variable is still in the query scope after the function that sets it returns).

If you later call a function using the variable a, you could get bad results:

def computeBadScope(a:int):
    return TableTools.emptyTable(10).update("X=i","Y=a+X")

This query will execute because a is already in the query scope from an earlier call to compute(). However, the a set by compute may be totally different than the argument to computeBadScope(), which will produce unintended results.

Groovy

Variables that are in the global Groovy binding are automatically added to the QueryScope, as in the following query:

z=3
result2 = emptyTable(10).update("X=i","Y=(int)f.call(z,X)")

If you are in a function, the variables are not automatically added to the binding, so this query will not work:

f = {int a, int b -> a*b}

The variables can be explicitly added to the binding as in this example:

// Variable "a" is not in the global Groovy binding, so it must be added to the QueryScope.
compute = { int a ->
   QueryScope.addParam("a",a)
   return emptyTable(10).update("X=i", "Y=X+a")
}
result1 = compute(10)

Note

Variables defined without "def" inside closures are added to the global Groovy binding, so they are automatically available in the QueryScope:

y = { ->
    b=3
    return emptyTable(10).update("X=i","Y=b=X")
}
result3 = y()
result4 = = emptyTable(10).update("Z=b")

However, Groovy variables defined with "def" are not part of the global Groovy binding, so they are not available, unless they are added to the QueryScope.

So the following query does not work:

def v1 = 4
result5 = emptyTable(10).update("V1=v1")

The following query does work:

v1 = 4
result5 = emptyTable(10).update("V1=v1")

Groovy variables defined with a type are also not part of the global Groovy binding, so they are not available, unless they are added to the QueryScope.

So the following query does not work:

int v2 = 4
result5 = emptyTable(10).update("V2=v2")

The following query does work:

v2 = 4
result5 = emptyTable(10).update("V2=v2")