Vectorization and the recipe paradigm

Deephaven's query engine uses vectorized operations and a declarative "recipe" paradigm to achieve high performance on both static and real-time data. This guide explains the technical foundations of this approach and why it matters for your queries.

This guide starts with the recipe model. Instead of writing step-by-step instructions that process data one element at a time, you describe the result you want, like a recipe that describes the finished dish. The engine works out how to compute that result and keeps it up to date as data changes. The guide then covers:

  • How the engine runs recipes over chunks of data in compiled code.
  • What that means for memory use.
  • Common patterns, when loops are still the right tool, and performance best practices.

The paradigm shift: Imperative vs declarative

Traditional programming: Imperative SISD

In traditional programming, you write imperative code that processes one data element at a time. This is Single Instruction, Single Data (SISD): each instruction operates on a single value:

This approach:

  • Executes instructions sequentially.
  • Processes one data element per instruction.
  • Requires explicit loops for multiple elements.
  • Creates intermediate objects for each value.

Deephaven: Declarative and chunked

Deephaven uses a declarative approach. You describe the columns you want, and the engine computes them in chunks. A chunk is a contiguous block of typed values from one column, such as 4,096 doubles. The engine applies each formula to a whole chunk in compiled code.

For primitive columns, the engine reads the values as primitives. It doesn't box them: boxing wraps each value in an object, such as an Integer.

This page uses vectorization to mean processing a column in chunks with compiled code, rather than one boxed value at a time. It is related to, but not the same as, CPU-level vectorization.

See emptyTable and update for more details.

i is a special variable that holds each row's index.

This approach:

  • Specifies what to compute, not how.
  • Processes data in optimized chunks (vectorization).
  • Avoids intermediate objects.

The recipe paradigm: How it works

Recipes are specifications

When you write a Deephaven query:

See timeTable for more details.

You're creating a specification (recipe) that says "Y should always equal X times 2". The engine computes Y for the existing rows when the query runs and keeps it up to date as t1 changes. You don't write the loop or the update logic.

Dependency tracking

For the t1 and t2 query in Recipes are specifications, where t2 adds Y = X * 2 to t1, the engine builds a Directed Acyclic Graph (DAG) of dependencies:

A table ticks when an update cycle adds, removes, or modifies its rows. The table update model describes the full cycle. Each time t1 ticks, this happens:

  1. New rows arrive in t1.
  2. t1 notifies t2, which registered as a listener when it was created.
  3. The engine computes Y for the new rows.
  4. The engine propagates the updates through the DAG.

Update propagation example

This example adds a cumulative sum to a ticking table:

See updateBy and CumSum for more details.

Watch the source and result tables in the UI. Every second:

  • A new row arrives in source.
  • The engine computes XSquared for the new row.
  • The engine updates SumX for the new row.

Write the recipe once, and it keeps the result up to date for as long as the table is live.

Why recipes enable real-time

Real-time processing is where the recipe paradigm shines: it makes keeping results current straightforward. Compare a loop with a recipe, both reading the ticking source table from the update propagation example.

Loop approach (doesn't work for real-time)

Imperative loops can't keep up with ticking data without significant additional infrastructure. A loop executes once and stops. You need to build your own subscription and recomputation logic.

Recipe approach (recomputes as data ticks)

Incremental computation

For many operations, including update, the engine doesn't recompute the whole table on each update. It processes only the rows that changed.

In the update propagation example, when new rows arrive in source:

  • The engine processes only the new rows.
  • It evaluates every formula for those rows.
  • It appends the results to the output columns.
  • Nothing else is recomputed.

For modified rows, the engine recomputes only the affected rows and passes the changes to downstream tables through the DAG.

What counts as an affected row depends on the operation. A cumulative sum recomputes every row after a modified row. A filter that depends on another ticking table, such as whereIn, can re-evaluate every row when that table changes.

Real-world example: Chained time calculations

This example chains three tables, each built with update on the one before, to show that a change propagates through the whole chain:

This example illustrates several key concepts:

  • Declarative recipes — Each update specifies what to compute, not how to loop.
  • Automatic propagation — All three tables (t1, t2, t3) update every second.
  • Chained operations — Tables build on each other through the DAG.
  • Real-time execution — Each new row gets all of these columns computed as it arrives.

D3 and D4 both equal 2,000,000,000 nanoseconds (2 seconds), because TS3 and TS4 are two ways to add 2 seconds to Timestamp.

Real-world example: Live rolling averages

This example keeps a 10-row rolling average price for each symbol in a ticking table of simulated trades:

See RollingAvg for more details.

This query:

  • Processes streaming trade data.
  • Maintains separate rolling averages per symbol.
  • Updates automatically as new data arrives.
  • Is extremely difficult to implement with loops.

How the engine executes formulas

This section covers how Deephaven stores columns, how the engine runs a formula over chunks in compiled code, and how that relates to CPU-level vectorization.

Why the recipe approach is faster

The recipe approach avoids the per-value overhead of Groovy loops for data-processing work:

  1. Vectorization — The engine processes data in batches (chunks), paying per-call overhead once per chunk instead of once per value.
  2. No dynamic Groovy calls or boxing — Computation stays in generated, type-specialized Java code.
  3. Better memory access — Sequential columnar reads are cache-friendly.
  4. Parallelization — The engine can split work across cores.

Columnar storage

Deephaven stores data by column rather than by row.

Row-oriented (like lists of maps):

Columnar (like Deephaven):

  • Stores column X values next to each other in memory.
  • Gives excellent cache locality.
  • Gives the just-in-time (JIT) compiler a chance to use SIMD instructions.

The chunk architecture

Deephaven moves data in Chunk objects, the contiguous blocks of typed values from one column described in Deephaven: Declarative and chunked.

When you write:

The engine:

  1. Reads column X in chunks (e.g., 4,096 values at a time).
  2. Applies the formula to each value in the chunk, in compiled code.
  3. Writes results to column Y in chunks.

Because each step works on a whole chunk, the engine pays the per-call overhead of reading, computing, and writing once per chunk rather than once per row.

Query compilation

Under the hood, Deephaven:

  1. Parses your query string into an Abstract Syntax Tree (AST).
  2. Analyzes the AST to determine dependencies and types.
  3. Generates Java code for the formula.
  4. Compiles the generated code.
  5. Executes the compiled code on chunks of data.

The one exception to these steps is a formula whose right-hand side is only the name of an existing column, such as Y = X. Deephaven treats it as a column reference, skips parsing and code generation, and compiles no code. Deephaven compiles every other expression, including Y = X * 2.

For example, if X is an int column, "Y = X * 2" becomes code shaped like this conceptual sketch. The real generated class has more parts, but the shape is the same: one pass over each chunk that applies the formula to every value.

This compiled code:

  • Makes no dynamic Groovy calls and boxes no primitive values.
  • Runs as Java Virtual Machine (JVM) bytecode that the JIT compiler can optimize.

JIT compilation

The JVM uses JIT compilation to optimize hot code paths. For Deephaven queries:

  1. Initial execution — The JVM runs the formula's compiled bytecode in its own bytecode interpreter, without dynamic Groovy calls.
  2. Profiling — The JVM identifies hot methods.
  3. Compilation — The JIT compiler compiles hot methods to native code.
  4. Optimization — The compiler can apply optimizations such as loop unrolling and SIMD instructions.

This means:

  • The first chunks may run more slowly, before the JIT has compiled the formula's hot methods to native code.
  • Later chunks run the optimized native code.
  • Long-running queries benefit most.

CPU-level vectorization

Modern CPUs have special instructions that operate on multiple data elements simultaneously. For example, instead of performing one addition at a time:

With a SIMD instruction, a CPU can do:

Deephaven's engine doesn't emit these SIMD instructions itself. The JVM's JIT compiler can use them when it compiles hot loops, but whether it does is up to the JVM. Deephaven's columnar storage and its code specialized for each chunk type give the JIT tight loops over primitive values, which are the loops it can vectorize.

Memory efficiency

No intermediate objects

A Groovy loop or collect call creates one boxed object per value:

The recipe approach stores values in primitive arrays, without boxing each value:

ii is the same row index as i, as a long, which keeps ii * ii from overflowing an int at 1,000,000 rows.

Column sharing

A ColumnSource is the object that holds a column's data, and a RowSet is the set of row keys a table contains. update and where reuse the source table's ColumnSource objects instead of copying them:

See where for more details.

  • t2 adds only the new Y column. It reuses t1's RowSet because t2 contains the same rows as t1.
  • t3 reuses X with a new, filtered RowSet.

Common patterns

Pattern: Element-wise operations

This query computes the length of the hypotenuse for each row from two columns:

The engine computes Z the way The chunk architecture describes: it reads X and Y in chunks, applies the compiled formula, and writes Z in chunks, with no Groovy calls and no intermediate objects.

Pattern: Conditional operations

This query labels each row with a category chosen by a ternary expression:

The ternary operator compiles to generated Java code, with no Groovy calls.

Pattern: Cross-row operations

Cumulative and rolling calculations depend on earlier rows. A formula can read a fixed offset such as X_[i-1], but it can't keep a running total or a window state correct as rows are added and modified.

updateBy keeps that state in the engine and updates it incrementally. The update propagation example uses CumSum, and the live rolling averages example uses RollingAvg. Doing the same with loops would take much more code and would lose automatic real-time propagation.

When to use loops

Valid use case: Data extraction

This loop reads each row of a table and prints it in Groovy:

This is extraction, not transformation. The data is leaving Deephaven.

Valid use case: Control flow

This loop creates one filtered table per symbol:

You're using loops to control table creation, not to transform table data.

Invalid use case: Column transformations

Don't compute column values in a loop. As the loop approach shows, the results end up in a Groovy list rather than a table, and the loop doesn't run again when data ticks. Use update instead.

Performance best practices

1. Prefer built-in expressions over Groovy closures

A compiled formula takes the same chunked path whether it is simple arithmetic or a chain of ternaries. What costs more is calling Groovy code from a formula:

❌ Slow — Calls a closure for every row:

✅ Fast — Stays in compiled code:

See User-defined functions for how to call closures from query strings when you need them.

2. Use appropriate operations

For rolling and cumulative calculations, use updateBy, as in Pattern: Cross-row operations.

For aggregations, use dedicated methods such as avgBy:

3. Filter early

Filter before you add computed columns, so later formulas run only on the rows you keep:

The where keeps 99,999 of the 1,000,000 rows, so the engine computes Y for only those rows.

Key takeaways

  1. Think declaratively — Specify what to compute, not how to iterate.
  2. Write recipes for real-time data — Declarative queries update automatically.
  3. Let chunks do the work — Chunked operations process many values per call.
  4. Prefer built-in expressions — Formulas that don't call closures stay in generated, type-specialized Java code.
  5. Use loops for extraction, not transformation — Loop to get data out of a table, and compute column values with query operations.

The paradigm shift:

  • Old way: "For each row, multiply X by 2 and store in Y".
  • Deephaven way: "Y should always equal X times 2".

This shift unlocks:

  • High performance through vectorization.
  • Automatic real-time updates.
  • Efficient memory usage.
  • Cleaner, more maintainable code.