Use Numba in Deephaven queries

This guide shows how to use Numba in Deephaven Python queries. The first sections cover the Numba basics the examples rely on: JIT compilation, Numba's decorators, lazy and eager compilation, and how @jit, @vectorize, and @guvectorize differ. Use Numba functions in query strings covers how Deephaven calls Numba functions in table operations and the rules those functions must follow.

Numba is an open-source just-in-time (JIT) compiler for Python. It uses LLVM to translate portions of Python code into optimized machine code, which can make your queries faster and more responsive.

A Python function in a query string runs for every row Deephaven computes, including the new and modified rows each time a real-time table updates. A faster function speeds up every update cycle.

Note

Numba is an optional package, and whether you already have it depends on how you run Deephaven:

  • Since 41.3, the deephaven-core and deephaven-server pip packages don't install it.
  • The ghcr.io/deephaven/server Docker images include it from version 43.0. The 41.3 through 42.x images don't.

Each example on this page runs pip install numba first, which does nothing if Numba is already installed. For the fastest query string calls, install Numba before Deephaven starts, as Use Numba functions in query strings explains.

What is just-in-time (JIT) compilation?

JIT compiles code into optimized machine code at runtime. With Numba, you mark which functions to compile at runtime. Compiling code blocks into optimized machine code adds some overhead, but subsequent function calls can be much faster. JIT is most useful for functions that are complex, large, or called many times.

Caution

JIT does not guarantee faster code. Numba can't optimize some code well, and that code may run slower.

Numba decorators and compilation

Numba uses decorators to specify which functions to JIT compile. Common Numba decorators include @jit, @vectorize, and @guvectorize. See Numba's documentation for more details.

@jit and @vectorize differ in what they're for:

  • @jit is a general-purpose decorator that can optimize a wide variety of functions.
  • @vectorize lets you write a function that takes scalars, and Numba applies it element-wise to arrays.

@jit vs @vectorize shows this difference in action, and @vectorize vs @guvectorize covers how @guvectorize differs from @vectorize.

The following example uses @jit to compile function_one and @vectorize to compile function_two, then calls each one. function_one takes two scalars. function_two is written for scalars, and Numba applies it element-wise to the two arrays.

Note

Since Numba 0.59, @jit compiles in nopython mode by default. Nopython mode produces machine code that runs without the Python interpreter. Numba no longer falls back to the slower object mode when it can't compile a function this way. Passing nopython=True, or using the equivalent @njit decorator, makes this explicit.

Lazy compilation

A function signature lists the input and output data types a compiled function accepts, such as double(double, int64) in the earlier function_two example. When you use a Numba decorator such as @jit or @vectorize without a function signature, Numba waits until the first call to compile the function. This is called lazy compilation. With lazy compilation, Numba:

  • Infers the argument types at call time.
  • Generates optimized code for those types.
  • Compiles a separate specialization for each new combination of input types.

The example below uses lazy compilation on the function func. The first call is slow because Numba compiles func to machine code before running it. The second call reuses the compiled code and runs much faster.

Eager compilation

When you use a Numba decorator such as @jit or @vectorize with a function signature, Numba compiles the function when Python defines it. This is called eager compilation.

The example below uses eager compilation on the function func by specifying two function signatures. Numba compiles func when Python defines it, so even the first call on each input type skips compilation and runs about as fast as the second call.

Caution

Eager compilation creates functions that only support specific function signatures. Arguments that Numba can't convert to one of the declared signatures raise an error.

@jit vs @vectorize

The difference between @jit and @vectorize shows up when you pass arrays to a function compiled with a scalar signature. In the following example, both decorators compile the same addition function with the scalar signature int64(int64, int64). The @jit call is wrapped in try/except so the example prints the error instead of stopping:

Functions created with @vectorize apply their scalar signature element-wise to arrays, so vectorize_add_arrays prints [ 5 7 9 11 13]. A @jit function compiled with a scalar signature such as int64(int64, int64) rejects array arguments, so the call to jit_add_arrays raises a TypeError because Numba has no compiled definition for array arguments. The lazily compiled @jit function in Lazy compilation accepts arrays because Numba compiles it for the argument types it receives.

@vectorize vs @guvectorize

The @vectorize decorator turns a Python function that takes scalar arguments into a NumPy ufunc. Without Numba, a fast ufunc has to be written in C.

The @guvectorize decorator creates generalized ufuncs, which differ from @vectorize functions in these ways:

  • They can work on any number of elements of the input arrays at once.
  • They can take and return arrays of different dimensions.
  • They don't return their result. Instead, the function fills in an output array that it takes as its last parameter.

When you call a @guvectorize function, you can leave out the output argument, and NumPy creates the output array and returns it.

The following example defines g, a @guvectorize function that adds a scalar to every element of an array, and calls it on [0, 1, 2, 3, 4] with the scalar 2.

The @guvectorize decorator in this example takes two arguments:

  • The first, [(int64[:], int64, int64[:])], is a list of type signatures like the one passed to @vectorize. It tells Numba that the function takes three parameters: a one-dimensional int64 input array (int64[:]), an int64 input scalar, and a one-dimensional int64 output array.
  • The second, "(n),()->(n)", is the layout signature. In it, (n) before the arrow is an n-element one-dimensional input array, () is a scalar input, and (n) after the arrow is the n-element one-dimensional output array, which the function fills in.

Use Numba functions in query strings

You call a Numba function in a query string by name, like any other Python function. Deephaven checks whether Numba is installed when the server starts, and the result decides how Deephaven calls the function.

Deephaven treats a @jit function like a plain Python function, whether or not Numba was installed at startup. Each call receives one row's values.

When Numba is installed at startup, these rules apply to @vectorize and @guvectorize functions in query strings:

  • For both decorators, Deephaven reads argument and return types from the function's Numba signature, so the function must have exactly one explicit signature. A function with no signature or with several signatures, such as the two-signature func in Eager compilation, raises an error.
  • Deephaven passes a @vectorize function whole chunks of column data as arrays, so one call computes many rows. A @guvectorize function still receives one row's values per call.
  • A @vectorize function must be the whole formula, as in "C = f(A, B)". You can't use it inside a larger expression or cast its result.
  • Each argument to a @vectorize function must be a column, a variable, or a constant.

If you install Numba with pip in a running session instead, Deephaven treats @vectorize and @guvectorize functions like plain Python functions until the server restarts. Each call receives one row's values, and Deephaven takes the column types from type hints, not from the Numba signature.

Important

To install Numba before the server starts, use a ghcr.io/deephaven/server image at version 43.0 or later, or build a custom Docker image that adds it. If you run Deephaven from the deephaven-server pip package, install Numba in the same Python environment before you start the server.

Compare @jit and @vectorize in a query

The following example compares the performance of @jit and @vectorize on Deephaven tables. It defines a column addition function and a cubic polynomial function, each in three versions: plain Python, @jit, and @vectorize. It then times how long each version takes to create a new column in a 625,000-row table.

If this example's pip install is what installs Numba, the @vectorize functions run like plain Python functions, and Deephaven takes their column types from the -> int and -> float type hints. To time chunked @vectorize calls, run the example on a server that had Numba installed at startup.

How much either decorator speeds up a formula depends on the workload, so benchmark your own queries before you decide to use Numba.

Use @guvectorize in a query

The following example calls a @guvectorize function in an update operation. The plain Python function array_from_val builds the array column Y, and g, the function from @vectorize vs @guvectorize, adds each row's X to it.

This version of g also has the return type hint -> npt.NDArray[np.int64]. Deephaven uses that type hint for the type of column Z when Numba was installed in the running session rather than at startup.

Each row of Z is the array in Y with that row's X added to every element.