Initialization and updates
Deephaven runs your script in two phases: initialization and real-time updates. Ticking tables, which update as new data arrives, behave differently in each phase. This guide explains the two phases and covers safe patterns for reading ticking tables from your script, from listeners, and from threads you start yourself.
Deephaven table operations define relationships that the engine keeps up to date as data arrives. They don't run just once over the data that exists when you create them. That's what makes Deephaven code different from an ordinary Groovy script, and it's why a value you read right after you create a table from an external source may not be there yet.
How Deephaven executes your script
Deephaven execution has two phases:
- Initialization — Your script runs, creating tables and building the dependency graph (DAG). Tables from external sources, such as Kafka, may not have data yet.
- Real-time updates — Data flows in continuously. The engine propagates changes through the DAG. Listeners fire.
The engine component that processes real-time updates is the update graph. It works in update cycles: in each cycle, it applies new data from sources and propagates the changes through the DAG. For more details on how updates propagate, see Incremental update model.
Deephaven runs each console command and Application Mode script while holding the update graph's exclusive lock, so no update cycle can run while your script runs. This has two consequences:
- Reads in your script see a consistent table. Your script doesn't need a lock or a snapshot for consistency.
- Data from external sources, such as a Kafka consumer, can't arrive until the script finishes, or until it calls
awaitUpdate, which releases the lock while it waits. A read right after you create such a table returns nothing.
Code on threads you start yourself doesn't hold this lock. See Use locking for reads from your own threads.
Safe patterns
Let the engine do the work
The safest approach is to express your logic as Deephaven table operations rather than reading values into Groovy variables. The engine keeps every table operation consistent as data ticks.
Use awaitUpdate to wait for data
In Application Mode scripts, which run when the server starts, use awaitUpdate to wait for external data before proceeding:
awaitUpdate can occasionally return early on a spurious wakeup, so the example loops until the table has rows. You can specify a timeout: trades.awaitUpdate(5000) returns false if no update occurs within 5 seconds.
Use snapshot for point-in-time extraction
To keep a static copy of a ticking table's current data, use snapshot. This example starts the time table 10 seconds in the past, so it already has rows when the snapshot runs:
Use snapshotWhen for periodic extraction
To extract data at controlled intervals, use snapshotWhen:
Use listeners for ongoing updates
For reacting to changes, use table listeners:
Caution
Keep listener execution fast. Until a running listener returns, the current update cycle can't finish and the next one can't start.
A listener that keeps state, such as a running total, usually needs to start from the rows the table already has. The InstrumentedTableUpdateListenerAdapter above can't replay those rows. There are two ways to start from them:
- Read the existing rows first: Read them in the script, for example from a
snapshot, then callsource.addUpdateListener(listener). The script holds the exclusive lock, so no update cycle runs in between and the listener doesn't miss any rows. - Replay the existing rows: Extend
ShiftObliviousInstrumentedListenerAdapterinstead. Register it withsource.addUpdateListener(listener, true). Thetrueargument replays the existing rows as the listener's first update, as Replay data shows.
Use locking for reads from your own threads (advanced)
Code on a thread you start yourself doesn't hold the exclusive lock that script code runs under. When several reads on such a thread must see the same update cycle, hold the update graph's shared lock around all of them:
The thread can't take the shared lock until the script finishes and releases the exclusive lock. Don't call the thread's join method in the same script, or the script hangs.
Caution
Keep lock duration minimal. While your thread holds the shared lock, no update cycle can run.
What not to do
- Don't read ticking tables from a thread you started unless you hold a lock. Script code already runs under the exclusive lock, but code on other threads doesn't. Hold the shared lock, as shown in Use locking for reads from your own threads, or read from a
snapshot. - Don't read undeclared ticking tables inside a listener. The engine doesn't guarantee that other tables have finished updating for the current cycle. If the listener needs another table, register it as a dependency.
- Don't hold locks during long computations. On a thread you start yourself, extract data quickly inside
doLocked, then process after the block. Script code holds the exclusive lock for its whole run, so run long computations on your own thread rather than in the script itself. - Don't mix data from different update cycles. Row data and row keys saved from an earlier cycle may no longer match the table, because rows can be removed, modified, or shifted. Instead:
- To carry information to the next cycle, store only computed results, not raw row data.
- To compare before and after values within one update, read the previous values of the modified rows, as the modified-rows example in Table listeners shows.
Related documentation
- Table listeners — Full guide to listeners in Groovy
- Incremental update model — How updates work internally
- Synchronization and locking — Advanced locking