Vectorization and the recipe paradigm
Deephaven's query engine uses vectorized operations and a declarative "recipe" paradigm to achieve high performance on both static and real-time data. This guide explains the technical foundations of this approach and why it matters for your queries.
This guide starts with the recipe model. Instead of writing step-by-step instructions that process data one element at a time, you describe the result you want, like a recipe that describes the finished dish. The engine works out how to compute that result and keeps it up to date as data changes. The guide then covers:
- How the engine runs recipes over chunks of data in compiled code.
- What that means for memory use.
- Common patterns, when loops are still the right tool, and performance best practices.
The paradigm shift: Imperative vs declarative
Traditional programming: Imperative SISD
In traditional programming, you write imperative code that processes one data element at a time. This is Single Instruction, Single Data (SISD): each instruction operates on a single value:
This approach:
- Executes instructions sequentially.
- Processes one data element per instruction.
- Requires explicit loops for multiple elements.
- Creates intermediate objects for each value.
Deephaven: Declarative and chunked
Deephaven uses a declarative approach. You describe the columns you want, and the engine computes them in chunks. A chunk is a contiguous block of typed values from one column, such as 4,096 doubles. The engine applies each formula to a whole chunk in compiled code.
For primitive columns, the engine reads the values as primitives. It doesn't box them: boxing wraps each value in an object, such as an Integer.
This page uses vectorization to mean processing a column in chunks with compiled code, rather than one boxed value at a time. It is related to, but not the same as, CPU-level vectorization.
See emptyTable and update for more details.
i is a special variable that holds each row's index.
This approach:
- Specifies what to compute, not how.
- Processes data in optimized chunks (vectorization).
- Avoids intermediate objects.
The recipe paradigm: How it works
Recipes are specifications
When you write a Deephaven query:
See timeTable for more details.
You're creating a specification (recipe) that says "Y should always equal X times 2". The engine computes Y for the existing rows when the query runs and keeps it up to date as t1 changes. You don't write the loop or the update logic.
Dependency tracking
For the t1 and t2 query in Recipes are specifications, where t2 adds Y = X * 2 to t1, the engine builds a Directed Acyclic Graph (DAG) of dependencies:
A table ticks when an update cycle adds, removes, or modifies its rows. The table update model describes the full cycle. Each time t1 ticks, this happens:
- New rows arrive in
t1. t1notifiest2, which registered as a listener when it was created.- The engine computes
Yfor the new rows. - The engine propagates the updates through the DAG.
Update propagation example
This example adds a cumulative sum to a ticking table:
See updateBy and CumSum for more details.
Watch the source and result tables in the UI. Every second:
- A new row arrives in
source. - The engine computes
XSquaredfor the new row. - The engine updates
SumXfor the new row.
Write the recipe once, and it keeps the result up to date for as long as the table is live.
Why recipes enable real-time
Real-time processing is where the recipe paradigm shines: it makes keeping results current straightforward. Compare a loop with a recipe, both reading the ticking source table from the update propagation example.
Loop approach (doesn't work for real-time)
Imperative loops can't keep up with ticking data without significant additional infrastructure. A loop executes once and stops. You need to build your own subscription and recomputation logic.
Recipe approach (recomputes as data ticks)
Incremental computation
For many operations, including update, the engine doesn't recompute the whole table on each update. It processes only the rows that changed.
In the update propagation example, when new rows arrive in source:
- The engine processes only the new rows.
- It evaluates every formula for those rows.
- It appends the results to the output columns.
- Nothing else is recomputed.
For modified rows, the engine recomputes only the affected rows and passes the changes to downstream tables through the DAG.
What counts as an affected row depends on the operation. A cumulative sum recomputes every row after a modified row. A filter that depends on another ticking table, such as whereIn, can re-evaluate every row when that table changes.
Real-world example: Chained time calculations
This example chains three tables, each built with update on the one before, to show that a change propagates through the whole chain:
This example illustrates several key concepts:
- Declarative recipes — Each
updatespecifies what to compute, not how to loop. - Automatic propagation — All three tables (
t1,t2,t3) update every second. - Chained operations — Tables build on each other through the DAG.
- Real-time execution — Each new row gets all of these columns computed as it arrives.
D3 and D4 both equal 2,000,000,000 nanoseconds (2 seconds), because TS3 and TS4 are two ways to add 2 seconds to Timestamp.
Real-world example: Live rolling averages
This example keeps a 10-row rolling average price for each symbol in a ticking table of simulated trades:
See RollingAvg for more details.
This query:
- Processes streaming trade data.
- Maintains separate rolling averages per symbol.
- Updates automatically as new data arrives.
- Is extremely difficult to implement with loops.
How the engine executes formulas
This section covers how Deephaven stores columns, how the engine runs a formula over chunks in compiled code, and how that relates to CPU-level vectorization.
Why the recipe approach is faster
The recipe approach avoids the per-value overhead of Groovy loops for data-processing work:
- Vectorization — The engine processes data in batches (chunks), paying per-call overhead once per chunk instead of once per value.
- No dynamic Groovy calls or boxing — Computation stays in generated, type-specialized Java code.
- Better memory access — Sequential columnar reads are cache-friendly.
- Parallelization — The engine can split work across cores.
Columnar storage
Deephaven stores data by column rather than by row.
Row-oriented (like lists of maps):
- Interleaves column
Xvalues withYvalues, so readingXskips overY. - Gives poor cache locality for column operations.
- Leaves little room for Single Instruction, Multiple Data (SIMD) instructions.
Columnar (like Deephaven):
- Stores column
Xvalues next to each other in memory. - Gives excellent cache locality.
- Gives the just-in-time (JIT) compiler a chance to use SIMD instructions.
The chunk architecture
Deephaven moves data in Chunk objects, the contiguous blocks of typed values from one column described in Deephaven: Declarative and chunked.
When you write:
The engine:
- Reads column
Xin chunks (e.g., 4,096 values at a time). - Applies the formula to each value in the chunk, in compiled code.
- Writes results to column
Yin chunks.
Because each step works on a whole chunk, the engine pays the per-call overhead of reading, computing, and writing once per chunk rather than once per row.
Query compilation
Under the hood, Deephaven:
- Parses your query string into an Abstract Syntax Tree (AST).
- Analyzes the AST to determine dependencies and types.
- Generates Java code for the formula.
- Compiles the generated code.
- Executes the compiled code on chunks of data.
The one exception to these steps is a formula whose right-hand side is only the name of an existing column, such as Y = X. Deephaven treats it as a column reference, skips parsing and code generation, and compiles no code. Deephaven compiles every other expression, including Y = X * 2.
For example, if X is an int column, "Y = X * 2" becomes code shaped like this conceptual sketch. The real generated class has more parts, but the shape is the same: one pass over each chunk that applies the formula to every value.
This compiled code:
- Makes no dynamic Groovy calls and boxes no primitive values.
- Runs as Java Virtual Machine (JVM) bytecode that the JIT compiler can optimize.
JIT compilation
The JVM uses JIT compilation to optimize hot code paths. For Deephaven queries:
- Initial execution — The JVM runs the formula's compiled bytecode in its own bytecode interpreter, without dynamic Groovy calls.
- Profiling — The JVM identifies hot methods.
- Compilation — The JIT compiler compiles hot methods to native code.
- Optimization — The compiler can apply optimizations such as loop unrolling and SIMD instructions.
This means:
- The first chunks may run more slowly, before the JIT has compiled the formula's hot methods to native code.
- Later chunks run the optimized native code.
- Long-running queries benefit most.
CPU-level vectorization
Modern CPUs have special instructions that operate on multiple data elements simultaneously. For example, instead of performing one addition at a time:
With a SIMD instruction, a CPU can do:
Deephaven's engine doesn't emit these SIMD instructions itself. The JVM's JIT compiler can use them when it compiles hot loops, but whether it does is up to the JVM. Deephaven's columnar storage and its code specialized for each chunk type give the JIT tight loops over primitive values, which are the loops it can vectorize.
Memory efficiency
No intermediate objects
A Groovy loop or collect call creates one boxed object per value:
The recipe approach stores values in primitive arrays, without boxing each value:
ii is the same row index as i, as a long, which keeps ii * ii from overflowing an int at 1,000,000 rows.
Column sharing
A ColumnSource is the object that holds a column's data, and a RowSet is the set of row keys a table contains. update and where reuse the source table's ColumnSource objects instead of copying them:
See where for more details.
t2adds only the newYcolumn. It reusest1'sRowSetbecauset2contains the same rows ast1.t3reusesXwith a new, filteredRowSet.
Common patterns
Pattern: Element-wise operations
This query computes the length of the hypotenuse for each row from two columns:
The engine computes Z the way The chunk architecture describes: it reads X and Y in chunks, applies the compiled formula, and writes Z in chunks, with no Groovy calls and no intermediate objects.
Pattern: Conditional operations
This query labels each row with a category chosen by a ternary expression:
The ternary operator compiles to generated Java code, with no Groovy calls.
Pattern: Cross-row operations
Cumulative and rolling calculations depend on earlier rows. A formula can read a fixed offset such as X_[i-1], but it can't keep a running total or a window state correct as rows are added and modified.
updateBy keeps that state in the engine and updates it incrementally. The update propagation example uses CumSum, and the live rolling averages example uses RollingAvg. Doing the same with loops would take much more code and would lose automatic real-time propagation.
When to use loops
Valid use case: Data extraction
This loop reads each row of a table and prints it in Groovy:
This is extraction, not transformation. The data is leaving Deephaven.
Valid use case: Control flow
This loop creates one filtered table per symbol:
You're using loops to control table creation, not to transform table data.
Invalid use case: Column transformations
Don't compute column values in a loop. As the loop approach shows, the results end up in a Groovy list rather than a table, and the loop doesn't run again when data ticks. Use update instead.
Performance best practices
1. Prefer built-in expressions over Groovy closures
A compiled formula takes the same chunked path whether it is simple arithmetic or a chain of ternaries. What costs more is calling Groovy code from a formula:
❌ Slow — Calls a closure for every row:
✅ Fast — Stays in compiled code:
See User-defined functions for how to call closures from query strings when you need them.
2. Use appropriate operations
For rolling and cumulative calculations, use updateBy, as in Pattern: Cross-row operations.
For aggregations, use dedicated methods such as avgBy:
3. Filter early
Filter before you add computed columns, so later formulas run only on the rows you keep:
The where keeps 99,999 of the 1,000,000 rows, so the engine computes Y for only those rows.
Key takeaways
- Think declaratively — Specify what to compute, not how to iterate.
- Write recipes for real-time data — Declarative queries update automatically.
- Let chunks do the work — Chunked operations process many values per call.
- Prefer built-in expressions — Formulas that don't call closures stay in generated, type-specialized Java code.
- Use loops for extraction, not transformation — Loop to get data out of a table, and compute column values with query operations.
The paradigm shift:
- Old way: "For each row, multiply
Xby 2 and store inY". - Deephaven way: "
Yshould always equalXtimes 2".
This shift unlocks:
- High performance through vectorization.
- Automatic real-time updates.
- Efficient memory usage.
- Cleaner, more maintainable code.
Related documentation
- Think like a Deephaven ninja
- Deephaven's design
- How Deephaven works: A mental model
- Table update model
- The live DAG
- Query engine parallelization
- Crash course: Recipes, not loops!
- Table operations
- Dedicated aggregations
- Filter table data
- Iterate over a column
- Rolling aggregations
- Ternary operator
- User-defined functions
- Special variables
- Arrays
avgByCumSumemptyTableepochNanosepochNanosToInstantRollingAvgtimeTableupdateupdateBywherewhereInTableJavadocUpdateByOperationJavadoc