Vectorization and the recipe paradigm
Deephaven's query engine uses vectorized operations and a declarative "recipe" paradigm to achieve high performance on both static and real-time data. This guide explains the technical foundations of this approach and why it matters for your queries.
This guide starts with the recipe model. Instead of writing step-by-step instructions that process data one element at a time, you describe the result you want, like a recipe that describes the finished dish. The engine works out how to compute that result and keeps it up to date as data changes. The guide then covers:
- How the engine runs recipes over chunks of data in compiled code.
- What that means for memory use.
- Common patterns, when loops are still the right tool, and performance best practices.
The paradigm shift: Imperative vs declarative
Traditional programming: Imperative SISD
In traditional Python, you write imperative code that processes one data element at a time. This is Single Instruction, Single Data (SISD): each instruction operates on a single value:
This approach:
- Executes instructions sequentially.
- Processes one data element per instruction.
- Requires explicit loops for multiple elements.
- Creates intermediate Python objects for each value.
Deephaven: Declarative and chunked
Deephaven uses a declarative approach. You describe the columns you want, and the engine computes them in chunks. A chunk is a contiguous block of typed values from one column, such as 4,096 doubles. The engine applies each formula to a whole chunk in compiled code, rather than to one value at a time in the Python interpreter.
This page uses vectorization to mean processing a column in chunks with compiled code, rather than one value at a time in an interpreter. It is related to, but not the same as, CPU-level vectorization.
See empty_table and update for more details.
i is a special variable that holds each row's index.
This approach:
- Specifies what to compute, not how.
- Processes data in optimized chunks (vectorization).
- Avoids intermediate Python objects.
The recipe paradigm: How it works
Recipes are specifications
When you write a Deephaven query:
See time_table for more details.
You're creating a specification (recipe) that says "Y should always equal X times 2". The engine computes Y for the existing rows when the query runs and keeps it up to date as t1 changes. You don't write the loop or the update logic.
Dependency tracking
For the t1 and t2 query in Recipes are specifications, where t2 adds Y = X * 2 to t1, the engine builds a Directed Acyclic Graph (DAG) of dependencies:
A table ticks when an update cycle adds, removes, or modifies its rows. The table update model describes the full cycle. Each time t1 ticks, this happens:
- New rows arrive in
t1. t1notifiest2, which registered as a listener when it was created.- The engine computes
Yfor the new rows. - The engine propagates the updates through the DAG.
Update propagation example
This example adds a cumulative sum to a ticking table:
See update_by and cum_sum for more details.
Watch the source and result tables in the UI. Every second:
- A new row arrives in
source. - The engine computes
XSquaredfor the new row. - The engine updates
SumXfor the new row.
Write the recipe once, and it keeps the result up to date for as long as the table is live.
Why recipes enable real-time
Real-time processing is where the recipe paradigm shines: it makes keeping results current straightforward. Compare a loop with a recipe, both reading the ticking source table from the update propagation example.
Loop approach (doesn't work for real-time)
Imperative loops can't keep up with ticking data without significant additional infrastructure. A loop executes once and stops. You need to build your own subscription and recomputation logic.
Recipe approach (recomputes as data ticks)
Incremental computation
For many operations, including update, the engine doesn't recompute the whole table on each update. It processes only the rows that changed.
In the update propagation example, when new rows arrive in source:
- The engine processes only the new rows.
- It evaluates every formula for those rows.
- It appends the results to the output columns.
- Nothing else is recomputed.
For modified rows, the engine recomputes only the affected rows and passes the changes to downstream tables through the DAG.
What counts as an affected row depends on the operation. A cumulative sum recomputes every row after a modified row. A filter that depends on another ticking table, such as where_in, can re-evaluate every row when that table changes.
Real-world example: Chained time calculations
This example chains three tables, each built with update on the one before, to show that a change propagates through the whole chain:
This example illustrates several key concepts:
- Declarative recipes — Each
updatespecifies what to compute, not how to loop. - Automatic propagation — All three tables (
t1,t2,t3) update every second. - Chained operations — Tables build on each other through the DAG.
- Real-time execution — Each new row gets all of these columns computed as it arrives.
D3 and D4 both equal 2,000,000,000 nanoseconds (2 seconds), because TS3 and TS4 are two ways to add 2 seconds to Timestamp.
Real-world example: Live rolling averages
This example keeps a 10-row rolling average price for each symbol in a ticking table of simulated trades:
See rolling_avg_tick for more details.
This query:
- Processes streaming trade data.
- Maintains separate rolling averages per symbol.
- Updates automatically as new data arrives.
- Is extremely difficult to implement with loops.
How the engine executes formulas
This section covers how Deephaven stores columns, how the engine runs a formula over chunks in compiled code, and how that relates to CPU-level vectorization.
Why the recipe approach is faster
The recipe approach avoids the overhead of the Python interpreter for data-processing work:
- Vectorization — The engine processes data in batches (chunks), paying per-call overhead once per chunk instead of once per value.
- No Python overhead — Computation stays in compiled code.
- Better memory access — Sequential columnar reads are cache-friendly.
- Parallelization — The engine can split work across cores.
Columnar storage
Deephaven stores data by column rather than by row.
Row-oriented (like Python lists of dicts):
- Interleaves column
Xvalues withYvalues, so readingXskips overY. - Gives poor cache locality for column operations.
- Leaves little room for Single Instruction, Multiple Data (SIMD) instructions.
Columnar (like Deephaven):
- Stores column
Xvalues next to each other in memory. - Gives excellent cache locality.
- Gives the just-in-time (JIT) compiler a chance to use SIMD instructions.
The chunk architecture
Deephaven moves data in Chunk objects, the contiguous blocks of typed values from one column described in Deephaven: Declarative and chunked.
When you write:
The engine:
- Reads column
Xin chunks (e.g., 4,096 values at a time). - Applies the formula to each value in the chunk, in compiled code.
- Writes results to column
Yin chunks.
Because each step works on a whole chunk, the engine pays the per-call overhead of reading, computing, and writing once per chunk rather than once per row.
Query compilation
Under the hood, Deephaven:
- Parses your query string into an Abstract Syntax Tree (AST).
- Analyzes the AST to determine dependencies and types.
- Generates Java code for the formula.
- Compiles the generated code.
- Executes the compiled code on chunks of data.
The one exception to these steps is a formula whose right-hand side is only the name of an existing column, such as Y = X. Deephaven treats it as a column reference, skips parsing and code generation, and compiles no code. Deephaven compiles every other expression, including Y = X * 2.
For example, if X is an int column, "Y = X * 2" becomes code shaped like this conceptual sketch. The real generated class has more parts, but the shape is the same: one pass over each chunk that applies the formula to every value.
This compiled code:
- Has no Python overhead.
- Runs as Java Virtual Machine (JVM) bytecode that the JIT compiler can optimize.
JIT compilation
The JVM uses JIT compilation to optimize hot code paths. For Deephaven queries:
- Initial execution — The JVM runs the formula's compiled bytecode in its own bytecode interpreter, not the Python interpreter.
- Profiling — The JVM identifies hot methods.
- Compilation — The JIT compiler compiles hot methods to native code.
- Optimization — The compiler can apply optimizations such as loop unrolling and SIMD instructions.
This means:
- The first chunks may run more slowly, before the JIT has compiled the formula's hot methods to native code.
- Later chunks run the optimized native code.
- Long-running queries benefit most.
CPU-level vectorization
Modern CPUs have special instructions that operate on multiple data elements simultaneously. For example, instead of performing one addition at a time:
With a SIMD instruction, a CPU can do:
Deephaven's engine doesn't emit these SIMD instructions itself. The JVM's JIT compiler can use them when it compiles hot loops, but whether it does is up to the JVM. Deephaven's columnar storage and its code specialized for each chunk type give the JIT tight loops over primitive values, which are the loops it can vectorize.
Memory efficiency
No intermediate Python objects
A Python loop or list comprehension creates one Python object per value:
The recipe approach stores values in primitive Java arrays inside the engine:
ii is the same row index as i, as a long, which keeps ii * ii from overflowing an int at 1,000,000 rows.
Column sharing
A ColumnSource is the object that holds a column's data, and a RowSet is the set of row keys a table contains. update and where reuse the source table's ColumnSource objects instead of copying them:
See where for more details.
t2adds only the newYcolumn. It reusest1'sRowSetbecauset2contains the same rows ast1.t3reusesXwith a new, filteredRowSet.
Common patterns
Pattern: Element-wise operations
This query computes the length of the hypotenuse for each row from two columns:
The engine computes Z the way The chunk architecture describes: it reads X and Y in chunks, applies the compiled formula, and writes Z in chunks, with no Python overhead and no intermediate objects.
Pattern: Conditional operations
This query labels each row with a category chosen by a ternary expression:
The ternary operator compiles to generated Java code with no Python interpreter overhead.
Pattern: Cross-row operations
Cumulative and rolling calculations depend on earlier rows. A formula can read a fixed offset such as X_[i-1], but it can't keep a running total or a window state correct as rows are added and modified.
update_by keeps that state in the engine and updates it incrementally. The update propagation example uses cum_sum, and the live rolling averages example uses rolling_avg_tick. Doing the same with loops would take much more code and would lose automatic real-time propagation.
When to use loops
Valid use case: Data extraction
This loop reads each row of a table and prints it in Python:
This is extraction, not transformation. The data is leaving Deephaven.
Valid use case: Control flow
This loop creates one filtered table per symbol:
You're using loops to control table creation, not to transform table data.
Invalid use case: Column transformations
Don't compute column values in a loop. As the loop approach shows, the results end up in a Python list rather than a table, and the loop doesn't run again when data ticks. Use update instead.
Performance best practices
1. Minimize cross-language calls
A compiled formula takes the same chunked path whether it is simple arithmetic or a chain of ternaries. What costs more is calling Python from a formula:
❌ Slow — Crosses into Python:
✅ Fast — Stays in compiled code:
Deephaven can auto-vectorize a Python function call in a formula, making one Python call per chunk of rows instead of one per row. It does this only when all of these hold:
- The function call is the entire formula, as in
Y = python_func(X). - The function is called by its bare name, as in
python_func(X). Deephaven doesn't batch a call through a module or object, such asnp.sqrt(X)orobj.func(X). - Every argument is a column, a variable, or a constant.
- The call passes exactly as many arguments as the function declares. Deephaven doesn't batch a call that leaves out parameters that have defaults, or that passes several values to
*args. - The call isn't wrapped in a cast. Deephaven doesn't batch
Y = (int)python_func(X). - The function returns a type Deephaven can batch: a primitive,
str,bool, a date-time that Deephaven converts to anInstant, an untyped object, or an array of one of these. A function with no return annotation qualifies.
This is a different meaning of "vectorization" from the one this page uses. Auto-vectorization batches the calls into Python, but it doesn't compile the function. The function body still runs in Python once for each value, so the call is slower than a built-in expression. For the cost of crossing into Python, see Calling Python from formulas.
2. Use appropriate operations
For rolling and cumulative calculations, use update_by, as in Pattern: Cross-row operations.
For aggregations, use dedicated methods such as avg_by:
3. Filter early
Filter before you add computed columns, so later formulas run only on the rows you keep:
The where keeps 99,999 of the 1,000,000 rows, so the engine computes Y for only those rows.
Key takeaways
- Think declaratively — Specify what to compute, not how to iterate.
- Write recipes for real-time data — Declarative queries update automatically.
- Let chunks do the work — Chunked operations process many values per call.
- Prefer built-in expressions — Formulas that don't call Python functions stay in compiled code.
- Use loops for extraction, not transformation — Loop to get data out of a table, and compute column values with query operations.
The paradigm shift:
- Old way: "For each row, multiply
Xby 2 and store inY". - Deephaven way: "
Yshould always equalXtimes 2".
This shift unlocks:
- High performance through vectorization.
- Automatic real-time updates.
- Efficient memory usage.
- Cleaner, more maintainable code.
Related documentation
- Think like a Deephaven ninja
- Deephaven's design
- How Deephaven works: A mental model
- Table update model
- The live DAG
- Query engine parallelization
- Crash course: Recipes, not loops!
- Table operations
- Dedicated aggregations
- Filter table data
- Iterate with
forloops - Ternary operator
- Python functions in query strings
- Rolling aggregations
- Special variables
- Arrays
- Time functions
avg_bycum_sumempty_tableiter_tuplerolling_avg_ticktime_tableupdateupdate_bywherewhere_inTablePydocdeephaven.updatebyPydoc