Formula evaluation errors

In Deephaven, a formula is an expression used to define calculations and transformations on table data. Formulas are string representations of operations that get executed by the query engine, commonly used in operations like update, select, and where. They follow a syntax similar to Java expressions and can reference column names, constants, functions, and other variables in the query scope.

For more information on formulas and their syntax, see the formula documentation.

This guide describes various types of errors that occur while evaluating a formula. You may also encounter formula syntax errors that need to be debugged.

Missing columns

A common mistake is referencing a column that does not exist. In the following example, the formula attempts to evaluate owner.toLowerCase():

However, the PersistentQueryStateLog does not contain an owner column. The formula parser does not know that a particular name should be a column rather than a variable name, class name, or other identifier. In the error message, the parser simply reports that it Cannot find variable or class owner:

You know, however, that you intended to reference a column, so you can query the table's definition to get a list of valid columns:

As the actual name of the column is Owner, update the query accordingly:

If you intended to use a variable, you can get a list of variable names in the session as follows:

Null pointers

A common programming error is to reference a field or call a function on a null object. The following example normalizes the ServerHost column to lowercase values:

However, the ServerHost is not always set, which results in a NullPointerException:

The simplest workaround in these cases is to check for null using a ternary operator and return null as follows:

Classes not in your Query Library

One of the most powerful features of Deephaven is its ability to call into arbitrary Java libraries. For example, we may want to use the Apache Commons Codec Hex class to decode hexadecimal numbers into byte arrays. The following example calls the Hex.decodeHex function on several values:

Instead of byte arrays, we get the following error:

Deephaven's formula parser does not consider all classes on the classpath when parsing a formula. You must add the formula to the query library in the current execution context. Manually adding classes to the query library prevents most changes in worker dependencies from affecting the behavior of queries and also permits better isolation between different queries. So, you must add the Hex class to the query library to get the desired result:

Errors/Exceptions from called functions

Deephaven permits calling arbitrary functions, and as such, the engine does not know the requirements for the function inputs. In this example, the Hex.decodeHex function is used on the string zazu, which is not a valid hexadecimal number:

The engine reports the following exception:

In these cases, you must examine the exception message produced by the function, the documentation of the function you are calling, and the input data to determine why it failed. In some cases, using a debugger is necessary to examine the state of the Java Virtual Machine (JVM).

Unsafe refreshing uses of ii/i/k

The Deephaven engine provides virtual row variables that you can use in formulas:

  • i - zero-based row index (integer)
  • ii - one-based row index (integer)
  • k - row key (often used in keyed tables)

These variables can cause errors in certain scenarios, particularly in refreshing (updating) tables. When a table refreshes, the query engine needs to determine which rows have changed and how to update dependent calculations. Using row position variables like i and ii in refreshing contexts can lead to incorrect results or failures because row positions may change during updates.

Using these variables to reference other rows (like the prior row) in an update formula can cause the query engine to fail because it cannot reliably track dependencies between rows during refreshes. The following example demonstrates this issue:

It fails with the following exception:

When this error occurs, you must use alternative query logic to achieve your desired result. For example, you may be able to group your data, apply a transformation, and then ungroup the data.