Threads for formula evaluation
The Deephaven query engine runs as a Java process and uses multiple threads to process requests. This guide shows which thread evaluates the formulas in your query operations, so you can interpret thread-dependent results and write formulas that work on whichever thread runs them. It covers the Deephaven server application, whether you run it from a Docker container, with the Gradle server-jetty-app:run task, or as a Core+ worker in Deephaven Enterprise. If you embed the query engine as a library, your application provides the threading model.
Thread pools
Most interaction with the Deephaven engine is via gRPC. The Java gRPC library first handles a gRPC request on a web-server thread. Depending on the request, the server then dispatches it to one of two thread pools:
- The serial executor handles requests using a single thread. Executing console commands, binding tables to console variables, and publishing an object to a named variable in the query scope through a scope ticket run on the serial executor. Its single thread provides a well-defined order for script code execution and changes to variables.
- The concurrent executor handles most other requests. These requests can run concurrently, and the default Periodic Update Graph provides the locking they need.
A query operation begins on one of these thread pools, but formula evaluation may move to another thread depending on the operation. The engine can also evaluate formulas on two more thread pools:
- The operation initialization thread pool parallelizes the work of creating a table.
- The Periodic Update Graph thread pool processes table updates.
Each thread's name is its pool's prefix, a hyphen, and a number, such as DeephavenApiServer-Scheduler-Serial-1:
| Thread pool | Thread name prefix | Default size | Configuration property |
|---|---|---|---|
| Serial executor | DeephavenApiServer-Scheduler-Serial | 1 | None |
| Concurrent executor | DeephavenApiServer-Scheduler-Concurrent | 4 | scheduler.poolSize |
| Operation initialization thread pool | OperationInitializationThreadPool-initializationExecutor | One thread per processor | OperationInitializationThreadPool.threads |
| Periodic Update Graph | PeriodicUpdateGraph-updateExecutor | One thread per processor | PeriodicUpdateGraph.updateThreads |
The Periodic Update Graph thread pool and the operation initialization thread pool can each be set to one thread. In that case, the engine doesn't hand work to that pool:
- With one update thread, the Periodic Update Graph processes updates on the thread that runs each update cycle,
PeriodicUpdateGraph.DEFAULT.refreshThread. - With one operation initialization thread, initialization work runs on the thread that calls the operation.
The following table summarizes which thread runs a formula in each kind of operation. The sections below explain each case.
| Operations | When the formula runs | Thread that runs it |
|---|---|---|
select, update, where | When the table is created | An operation initialization thread. The calling thread if the formula is serial or the pool has one thread. |
select, update, where | When the source table updates | A Periodic Update Graph thread. |
view, update_view | Each time something reads the value | The thread that reads the value. |
select and update
The select and update operations use the same threading behavior: both eagerly compute results during initialization or in response to a table update. Consider the following snippet executed from the console:
The with_serial method marks the Thr formula serial. The engine never evaluates a serial formula concurrently with itself, and it evaluates the rows one at a time, in table order. Because the engine doesn't parallelize a serial formula, the formula runs on the thread that calls update. A console command runs on the serial executor, so Thr holds DeephavenApiServer-Scheduler-Serial-1.
For comparison, let's remove with_serial and run the same query:
This formula is eligible for parallel evaluation, so the engine hands the work to the operation initialization thread pool, even for a single row. Thr holds a value such as OperationInitializationThreadPool-initializationExecutor-3.
Refreshing select and update
Each time a source table updates, an update graph evaluates the downstream effects. The default Periodic Update Graph evaluates them on its own thread pool. The following query keeps the first two rows of a time table that adds a row once per second:
In this case, the formula runs on a Periodic Update Graph thread, such as PeriodicUpdateGraph-updateExecutor-6.
view and update_view
Unlike select and update, the view and update_view operations compute a value only when something reads it, so the formula runs on whichever thread does the reading. For example, when another query operation reads the results, the formula runs on the thread or threads that operation uses:
The value of Thr in distinct_threads is DeephavenApiServer-Scheduler-Serial-1 — the thread that executed the select_distinct operation.
When you open thread_name in the web UI, though, Thr shows a different thread, such as DeephavenApiServer-Scheduler-Concurrent-4. The web UI reads the table through a Barrage subscription. The server takes the Barrage snapshot of a one-column table like this one on a concurrent executor thread. It can read a table with several columns in parallel on the operation initialization thread pool instead.
The engine recomputes the value each time the UI reads the cell, such as when you reload or scroll the table. Each read can run on a different thread, so the value can change every time you look at it.
where
Like select and update, the where operation evaluates its formula eagerly. Let's record each thread that evaluates the filter:
The printed set holds one or more operation initialization thread names, such as {'OperationInitializationThreadPool-initializationExecutor-1'}. A plain Python set is thread-safe here because a single add call on a built-in set is atomic. Other shared state must be thread-safe, or you must mark the filter serial, as described in Write formulas that work on any thread.
Refreshing where
The engine evaluates a refreshing where operation on the Periodic Update Graph thread pool:
Let's wait a few seconds for the table to tick, then print used_threads in a separate command. Don't call time.sleep in the same command to wait. Console code runs under the update graph's exclusive lock, so the table can't tick until the command finishes.
The output is a set of one or more Periodic Update Graph thread names, such as {'PeriodicUpdateGraph-updateExecutor-6'}.
Write formulas that work on any thread
A formula should not depend on which thread runs it. The engine can evaluate one formula on several threads at once, and it can re-evaluate a view or update_view formula later on a different thread. If a formula updates shared state, such as a counter or a collection, use one of these approaches:
- Make the shared state thread-safe.
- Mark a
select,update, orwhereformula or filter serial withwith_serialso the engine never evaluates it concurrently with itself.
You can't mark a view or update_view formula serial. Both take only formula strings, not Selectable objects. They compute values on whichever thread reads them, in any order, so make the shared state those formulas use thread-safe.
If several formulas in the same select, update, or where call share the state, also order them with Barriers, as described in Parallelizing queries. Formulas in different operations can run at the same time on different threads, so any state they share must be thread-safe.
Table operations in formulas
A formula can itself run table operations, as a partitioned table transform does. These operations run on the same threads as any other select or update formula, so they might not run on the thread that started them. They run in an ExecutionContext.
- The
ExecutionContextcontrols which variables and methods formulas can use, and it carries information about the user and query initialization. - Each thread has its own current context, so a worker thread does not automatically have the context your script was using.
- A user context is any context that is not systemic.
- The console runs your script in a systemic context, so a console script has no user context open unless you open one.
Which context the nested operations get depends on whether a user context is open when you call select or update:
- If no user
ExecutionContextis open,selectandupdateuse a newly created context. - If a user
ExecutionContextis open, the engine uses that context for evaluation, and table operations inside the formula work.
The newly created context uses the source table's update graph, so nested operations still run on the thread pools described above. It has no query scope, no query library, and no query compiler, so:
- Formulas cannot see your variables or use your imported classes and functions.
- Any table operation that compiles a formula or other code fails. This includes
where,select, andupdatewith a formula, and some exact and relational joins, time-series joins, aggregations, and sorts.
Partitioned table transform follows the same rule. It uses the enclosing context only when that context is a user context.
When a transform function runs table operations, create a user context with make_user_exec_ctx before you call transform. Create it outside the function, because a context created inside the function copies the limited context the function runs in. Then open it in one of these ways:
- Open it in a
withblock around thetransformcall. - Open it inside the function with a
with ctx:block, wherectxis the context you created outside the function.
Without a user context, the transform function's update call fails, because the newly created context has no query scope:
The error ends with No ExecutionContext registered, or current ExecutionContext has no QueryScope.
The following example opens a user context around the transform call instead:
Keep tables created in formulas alive
When a select or update formula on a refreshing table returns a LivenessReferent, such as a table, the result table keeps that object alive until its row is removed or its value is replaced.
You can't use view or update_view to create a column of tables or other LivenessReferent objects, because their results would not have a well-defined liveness scope. By default, the engine rejects such a column and tells you to use select or update instead.
Related documentation
- Aggregations in Deephaven
- Create a time table
- Deephaven's live Directed-Acyclic-Graph (DAG)
- Exact and relational joins
- Execution context
- Filters
- Formulas
- How to use liveness scopes
- Inexact, time-series, and range joins
- Multithreading: Synchronization, locks, and snapshots
- Parallelizing queries
- Partitioned tables
- Periodic Update Graph
- Query scope
- Sort table data
- Track processing time and measure latency
- What is Barrage?
BarrierConcurrencyControlempty_tableheadmergepartition_byselectselect_distincttime_tabletransformupdateupdate_viewviewwhereLivenessReferentJavadocmake_user_exec_ctxPydocSelectablePydoc