Multithreading: Synchronization, locks, and snapshots
Deephaven often runs on multiprocessor systems, where multiple threads read and change the same data. The UpdateGraph is the component that propagates table updates. Ticking tables change each time it runs an update cycle, and other threads may be reading those tables at the same time. Deephaven synchronizes that access with three mechanisms:
- A shared lock lets threads run table operations and read data while no update cycle is running.
- An exclusive lock gives one thread sole access. The UpdateGraph holds it while it propagates changes.
- An optimistic snapshotting mechanism retrieves data and runs table operations without holding a lock. It falls back to the shared lock only when it can't get a consistent result.
Note
The engine's locks and snapshots are the same in Python and Groovy. In Python, you use locks through the deephaven.update_graph module, as Locking from Python shows. The other examples on this page use Deephaven's Java API from Groovy. The condition and snapshot APIs they show have no Python wrapper.
Choosing between snapshots and locks
The right choice depends on the operation and on the thread that runs it. Use these guidelines:
- Skip these mechanisms for most code you run in the Deephaven console, because Deephaven holds the exclusive lock while it runs console commands.
- Use snapshots to read data from ticking tables, especially for a user interface. The Deephaven web IDE uses snapshots for this reason. A snapshot can run while the UpdateGraph is updating, and it often returns sooner than a thread could acquire a lock. From Python:
to_pandasreads a ticking table through the lock-free snapshotting mechanism when you use its defaultdtype_backend.snapshotreturns a static copy of a ticking table. Whileauto_lockingis enabled, it acquires the shared lock first, so on a background thread it waits for a running update cycle to finish.- The Snapshotting section shows the Java API for writing custom snapshots.
- Use the shared lock to run table operations from a thread that does not already hold a lock. The shared lock lets multiple threads run at once, so prefer it to the exclusive lock.
- Some operations, such as
whereandsort, initialize from a snapshot and don't need a lock. - Others, such as
updateandnatural_join, must run under a lock when any input table is refreshing. In Python,auto_lockingacquires it for you around each such operation. Hold it yourself only when several operations must see the same update cycle.
- Some operations, such as
- Use the exclusive lock in advanced cases that must wait for the UpdateGraph to process changes and that
await_updatecan't handle.
Query engine locks
The UpdateGraph's shared lock and exclusive lock synchronize access to Deephaven data. By default, the UpdateGraph is the Periodic Update Graph, which runs an update cycle at a regular interval. The UpdateGraph holds the exclusive lock for the whole update cycle, so no other thread can hold either lock while a cycle runs.
Acquire locks only when necessary and hold them for the shortest duration possible. The UpdateGraph cannot start an update cycle until every other thread has released them.
Shared lock
Hold the shared lock to run table operations, such as update and natural_join, from your own threads. In Python, auto_locking usually acquires it for you.
You can also hold the shared lock to read large amounts of data from tables. Snapshotting is the better choice when the read finishes before the next update cycle starts, because it doesn't block the UpdateGraph. A read that spans the start or end of an update cycle is inconsistent. After the attempts set in Configuration, the snapshotting mechanism falls back to the shared lock.
How long a thread waits for the shared lock depends on what the UpdateGraph and other threads are doing:
- During an update cycle, the thread waits until the cycle completes and the UpdateGraph releases the exclusive lock.
- Between cycles, if no other thread holds or is waiting for the exclusive lock, the thread acquires the shared lock without waiting.
- If another thread holds or is waiting for the exclusive lock, the thread waits. That other thread can be the UpdateGraph waiting to start its next cycle, or a running console command.
A thread waits behind a queued exclusive-lock request because a fair
ReentrantReadWriteLock
backs both locks, and a fair lock grants waiting threads the lock in roughly arrival order. The backing lock
is an implementation detail and may change.
Exclusive lock
Only one thread at a time can hold the exclusive lock, and no other thread can hold the shared lock while the exclusive lock is held. Deephaven also holds the exclusive lock automatically while it runs commands in the Deephaven console or an Application Mode startup script.
You rarely hold the exclusive lock yourself. Its main use is to wait for changes from the UpdateGraph, which this page covers after it shows how to acquire a lock.
Locking from Python
In Python, the deephaven.update_graph module
provides the shared and exclusive locks:
shared_lockandexclusive_lockare context managers that hold a lock for the duration of awithblock.shared_lockedandexclusive_lockedare decorators that run a function under a lock.
The module's auto_locking setting is True by default. While it is enabled:
- A table operation that needs the lock acquires the shared lock if any input table is refreshing.
- The operation skips the lock if the calling thread already holds a lock, is an UpdateGraph thread, or is marked safe for serial table operations. An UpdateGraph thread is a thread that the UpdateGraph uses to run update cycles and listeners.
- The operation releases the lock when it finishes.
As a result, a thread outside the console rarely needs to hold the lock for a single table operation. Hold the shared
lock yourself when several operations must see the same update cycle. In the following example, a
background thread holds one shared lock while it reads a ticking table and its
sum_by aggregation with
to_pandas. No update cycle can run between the two reads, so the two
DataFrames agree.
Background threads have no execution context by default, so the thread opens the script's context first:
The thread can't acquire the shared lock until the console finishes running this script and releases the exclusive
lock. It then prints Totals match: True to the console log.
Using locks from Java (Groovy)
From Java or Groovy, the preferred way to use the shared lock or the exclusive lock is as a
functional lock. You pass the code to
run under a functional lock as an argument, typically a lambda expression, and the functional lock acquires and releases the lock
around it. This simplifies the standard try/finally locking pattern shown in
Explicit locking and unlocking.
The doLocked method takes a
ThrowingRunnable and runs it while the thread holds the lock:
Run as one console command, this example prints []. The time table adds its first row on the next update cycle, which can't start until the command finishes.
To compute a value while holding the lock and return it to the caller, use
computeLocked, which takes a
ThrowingSupplier:
Both doLocked and computeLocked let the passed function throw checked exceptions, such as IOException and TimeoutException. A checked exception is any exception that is not a subclass of RuntimeException. Both methods rethrow it to the caller, which must catch or declare it in Java. Groovy doesn't require this, but catching it keeps the error handling explicit:
The catch block rethrows the IOException inside an UncheckedIOException, so this example ends with that error.
Explicit locking and unlocking
You can also acquire and release a lock explicitly with the
lock and
unlock
methods, but functional locks are safer. If you lock explicitly, call unlock in a finally block so the lock is
released even when an exception or error occurs. The
doLocked and
computeLocked methods do this for you.
The following example shows the correct way to use the shared lock with explicit locking:
Waiting for UpdateGraph changes with the exclusive lock
You can use the exclusive lock to wait for changes from the UpdateGraph.
An operation needs to wait when it depends on an UpdateGraph cycle that happens partway through it. For example,
suppose your query flushes data to a
DynamicTableWriter during initialization. The
output table does not show that data until the UpdateGraph runs an update cycle.
The most common way to wait is await_update. Calling it on the output table
acquires the exclusive lock and waits until that cycle has run, after which the flushed rows are visible. While it
waits, it releases the lock, so the UpdateGraph keeps running cycles.
Note
Deephaven doesn't support upgrading from the shared lock to the exclusive lock. Attempting it throws an exception.
Accordingly, you can't call await_update while your thread holds the shared lock.
When await_update isn't enough, an independent thread can hold the exclusive lock and wait on a
condition
that the UpdateGraph signals. Use this pattern when a thread that the UpdateGraph doesn't run must wait for a change or
notification that the UpdateGraph propagates. Pair it with a custom
listener. The pattern has three steps:
- Obtain a condition with
updateGraph.exclusiveLock().newCondition(), whereupdateGraphis the UpdateGraph you capture from the table or execution context before you start the independent thread. - In the independent thread, acquire the exclusive lock and
awaitthe condition.awaitreleases the lock while the thread waits and reacquires it before returning. - From code that the UpdateGraph runs, such as a listener's
onUpdatemethod, callrequestSignal. The UpdateGraph signals the condition while it holds the exclusive lock. A listener must not acquire either lock during an update cycle, and attempting it throws an exception. That's why the listener asks the UpdateGraph to signal the condition instead of signaling it directly.
The following Groovy example shows this pattern:
With the default number of update threads on a multicore
machine, the UpdateGraph runs the listener on one of its update threads, so the
Signaling condition... and Condition signaled. lines show a thread name such as
PeriodicUpdateGraph-updateExecutor-1. The following output comes from a server configured with a single
update thread, so those lines show
PeriodicUpdateGraph.DEFAULT.refreshThread:
The output shows the progress of the two threads:
myIndependentThreadacquires the exclusive lock and waits for the condition. Waiting releases the lock until the condition is signaled.- The UpdateGraph processes a
timeTabletick, which triggers the listener'sonUpdatemethod. onUpdaterequests a signal for the condition.- After the UpdateGraph finishes its cycle,
myIndependentThreadreturns fromawaitand resumes.
The time table adds its first row on the first update cycle after it is created, and later rows on five-second boundaries. The listener keeps running until you remove it from the table or it is garbage collected.
Snapshotting
A snapshot is a guaranteed-consistent result derived from data in Deephaven without necessarily acquiring a lock. Deephaven's snapshotting mechanism is a form of optimistic concurrency that helps to retrieve data safely, quickly, and without blocking the UpdateGraph.
ConstructSnapshot.callDataSnapshotFunction
runs a snapshot. It attempts the operation without acquiring a lock, then verifies that the UpdateGraph did not change
any data while the operation was running. If data may have changed while the operation was running, the snapshotting
mechanism retries the operation.
If the snapshot cannot complete successfully after a limited number of attempts, or if a failed attempt runs longer than a configured duration, the snapshotting mechanism acquires the shared lock to perform the operation reliably. The Configuration section lists the properties that set these limits.
Lock-free attempts don't always happen. callDataSnapshotFunction
skips them, and runs the snapshot function once, in two cases:
- The calling thread already holds the shared or exclusive lock. Console commands hold the exclusive lock, so a snapshot that you take from the console runs this way.
- The calling thread is an UpdateGraph thread, one of the threads that the UpdateGraph uses to run update cycles and listeners.
Obtaining snapshots
Obtaining a snapshot of tables or other data that the UpdateGraph updates takes two steps:
- Create
a
SnapshotControlobject to control snapshot validation. - Run the snapshot by passing the
SnapshotControland aSnapshotFunction, which holds the logic that must run on consistent data, tocallDataSnapshotFunction.
Creating the SnapshotControl
To obtain snapshots, first use
ConstructSnapshot.makeSnapshotControl
to create a SnapshotControl object, which tells the query engine how to verify that the snapshot result is correct.
The makeSnapshotControl parameters notificationAware, refreshing, and sources control this verification process.
| Parameter Name | Description |
|---|---|
notificationAware | Whether the snapshot result must capture every later notification from the sources. Set it to true when the snapshot initializes state that a listener then updates from those notifications. In most other cases, use false. |
refreshing | Whether the SnapshotControl needs to handle refreshing data. This must be true if any of the sources is refreshing. When it is false, the snapshot function runs once, with no lock and no consistency check. |
sources | A varargs parameter that must contain every source the snapshot function reads, including every table. |
Running a snapshot
To run a snapshot, pass three arguments to callDataSnapshotFunction:
- A log prefix.
- The
SnapshotControl. - The snapshot function, typically a lambda. It may run more than once, as described in Snapshotting.
The snapshot function receives two arguments, usePrev and beforeClockValue. usePrev says which values to read:
true: the snapshot is running during an update cycle before the sources finish updating, so the function must read previous values, the values as of the start of the current cycle.false: the function reads current values.
beforeClockValue is the logical clock value that the snapshotting mechanism read from the UpdateGraph before the attempt started. The logical clock changes when each update cycle starts and ends. The examples on this page don't use beforeClockValue.
Typically, a snapshot function retrieves data from several
ColumnSources for one or more
row keys, the keys that identify each row in a table. Consider the example
below, which retrieves data at row key 0 from two tables:
For each attempt, callDataSnapshotFunction handles the result as follows:
- If the function throws a
SnapshotUnsuccessfulExceptionor aCancellationException,callDataSnapshotFunctionrethrows it immediately without checking consistency. - If the function throws a
ConstructSnapshot.NoSnapshotAllowedException,callDataSnapshotFunctionmakes no further lock-free attempts and runs the function again under the shared lock. - After any other attempt that runs without the lock, the
SnapshotControlchecks whether the data the function read stayed consistent. If the data was inconsistent,callDataSnapshotFunctiondiscards the attempt, including any exception the function threw because it read inconsistent data, and runs the snapshot function again. - After the attempts reach the limits in Configuration,
callDataSnapshotFunctionruns the function under the shared lock. - If the data was consistent and the function returned
true, the snapshot succeeds. - If the data was consistent but the function returned
falseor threw an exception,callDataSnapshotFunctionthrows aSnapshotUnsuccessfulException, with any exception from the function as its cause. - When the function runs outside the lock-free attempts, returning
falseor throwing an exception makescallDataSnapshotFunctionthrow an exception.
Early stopping of inconsistent snapshots
A snapshot function that runs several operations can stop early when the data becomes inconsistent. Between
operations, call
ConstructSnapshot.concurrentAttemptInconsistent.
It returns true if the sources or the UpdateGraph's logical clock have changed in a way that makes
the current lock-free attempt inconsistent. It always returns false outside a lock-free attempt. When it returns
true, return false from the snapshot function to end the attempt early. callDataSnapshotFunction
then finds the data inconsistent and runs the function again.
In the following example, the snapshot function retrieves data from 100 columns in each of two tables and checks for consistency before it reads the second table:
Configuration
The following properties limit how many lock-free attempts a snapshot makes, and how long a failed attempt can run, before the snapshotting mechanism acquires the shared lock.
| Property | Default Value | Description |
|---|---|---|
ConstructSnapshot.maxConcurrentAttempts | 2 | Maximum number of lock-free snapshot attempts before acquiring the shared lock. |
ConstructSnapshot.maxConcurrentAttemptDurationMillis | 5000 | Maximum duration, in milliseconds, of a failed lock-free attempt before the snapshotting mechanism stops retrying and acquires the shared lock. |
Related documentation
- Deephaven's live Directed-Acyclic-Graph (DAG)
- Execution Context
- Incremental update model
- Parallelizing queries
- Periodic Update Graph
- Initialization and updates
- Listen to ticking tables
- Create a time table
- Application Mode configuration
await_updateDynamicTableWriternatural_joinsnapshotsortsum_bytime_tableto_pandasupdateviewwheredeephaven.update_graphPydocConstructSnapshotJavadocFunctionalLockJavadocUpdateGraphJavadoc