Multithreading: Synchronization, locks, and snapshots

Deephaven often runs on multiprocessor systems, where multiple threads read and change the same data. The UpdateGraph is the component that propagates table updates. Ticking tables change each time it runs an update cycle, and other threads may be reading those tables at the same time. Deephaven synchronizes that access with three mechanisms:

  • A shared lock lets threads run table operations and read data while no update cycle is running.
  • An exclusive lock gives one thread sole access. The UpdateGraph holds it while it propagates changes.
  • An optimistic snapshotting mechanism retrieves data and runs table operations without holding a lock. It falls back to the shared lock only when it can't get a consistent result.

Note

The engine's locks and snapshots are the same in Python and Groovy. In Python, you use locks through the deephaven.update_graph module, as Locking from Python shows. The other examples on this page use Deephaven's Java API from Groovy. The condition and snapshot APIs they show have no Python wrapper.

Choosing between snapshots and locks

The right choice depends on the operation and on the thread that runs it. Use these guidelines:

  • Skip these mechanisms for most code you run in the Deephaven console, because Deephaven holds the exclusive lock while it runs console commands.
  • Use snapshots to read data from ticking tables, especially for a user interface. The Deephaven web IDE uses snapshots for this reason. A snapshot can run while the UpdateGraph is updating, and it often returns sooner than a thread could acquire a lock. From Python:
    • to_pandas reads a ticking table through the lock-free snapshotting mechanism when you use its default dtype_backend.
    • snapshot returns a static copy of a ticking table. While auto_locking is enabled, it acquires the shared lock first, so on a background thread it waits for a running update cycle to finish.
    • The Snapshotting section shows the Java API for writing custom snapshots.
  • Use the shared lock to run table operations from a thread that does not already hold a lock. The shared lock lets multiple threads run at once, so prefer it to the exclusive lock.
    • Some operations, such as where and sort, initialize from a snapshot and don't need a lock.
    • Others, such as update and natural_join, must run under a lock when any input table is refreshing. In Python, auto_locking acquires it for you around each such operation. Hold it yourself only when several operations must see the same update cycle.
  • Use the exclusive lock in advanced cases that must wait for the UpdateGraph to process changes and that await_update can't handle.

Query engine locks

The UpdateGraph's shared lock and exclusive lock synchronize access to Deephaven data. By default, the UpdateGraph is the Periodic Update Graph, which runs an update cycle at a regular interval. The UpdateGraph holds the exclusive lock for the whole update cycle, so no other thread can hold either lock while a cycle runs.

Acquire locks only when necessary and hold them for the shortest duration possible. The UpdateGraph cannot start an update cycle until every other thread has released them.

Shared lock

Hold the shared lock to run table operations, such as update and natural_join, from your own threads. In Python, auto_locking usually acquires it for you.

You can also hold the shared lock to read large amounts of data from tables. Snapshotting is the better choice when the read finishes before the next update cycle starts, because it doesn't block the UpdateGraph. A read that spans the start or end of an update cycle is inconsistent. After the attempts set in Configuration, the snapshotting mechanism falls back to the shared lock.

How long a thread waits for the shared lock depends on what the UpdateGraph and other threads are doing:

  • During an update cycle, the thread waits until the cycle completes and the UpdateGraph releases the exclusive lock.
  • Between cycles, if no other thread holds or is waiting for the exclusive lock, the thread acquires the shared lock without waiting.
  • If another thread holds or is waiting for the exclusive lock, the thread waits. That other thread can be the UpdateGraph waiting to start its next cycle, or a running console command.

A thread waits behind a queued exclusive-lock request because a fair ReentrantReadWriteLock backs both locks, and a fair lock grants waiting threads the lock in roughly arrival order. The backing lock is an implementation detail and may change.

Exclusive lock

Only one thread at a time can hold the exclusive lock, and no other thread can hold the shared lock while the exclusive lock is held. Deephaven also holds the exclusive lock automatically while it runs commands in the Deephaven console or an Application Mode startup script.

You rarely hold the exclusive lock yourself. Its main use is to wait for changes from the UpdateGraph, which this page covers after it shows how to acquire a lock.

Locking from Python

In Python, the deephaven.update_graph module provides the shared and exclusive locks:

The module's auto_locking setting is True by default. While it is enabled:

  • A table operation that needs the lock acquires the shared lock if any input table is refreshing.
  • The operation skips the lock if the calling thread already holds a lock, is an UpdateGraph thread, or is marked safe for serial table operations. An UpdateGraph thread is a thread that the UpdateGraph uses to run update cycles and listeners.
  • The operation releases the lock when it finishes.

As a result, a thread outside the console rarely needs to hold the lock for a single table operation. Hold the shared lock yourself when several operations must see the same update cycle. In the following example, a background thread holds one shared lock while it reads a ticking table and its sum_by aggregation with to_pandas. No update cycle can run between the two reads, so the two DataFrames agree.

Background threads have no execution context by default, so the thread opens the script's context first:

The thread can't acquire the shared lock until the console finishes running this script and releases the exclusive lock. It then prints Totals match: True to the console log.

Using locks from Java (Groovy)

From Java or Groovy, the preferred way to use the shared lock or the exclusive lock is as a functional lock. You pass the code to run under a functional lock as an argument, typically a lambda expression, and the functional lock acquires and releases the lock around it. This simplifies the standard try/finally locking pattern shown in Explicit locking and unlocking.

The doLocked method takes a ThrowingRunnable and runs it while the thread holds the lock:

Run as one console command, this example prints []. The time table adds its first row on the next update cycle, which can't start until the command finishes.

To compute a value while holding the lock and return it to the caller, use computeLocked, which takes a ThrowingSupplier:

Both doLocked and computeLocked let the passed function throw checked exceptions, such as IOException and TimeoutException. A checked exception is any exception that is not a subclass of RuntimeException. Both methods rethrow it to the caller, which must catch or declare it in Java. Groovy doesn't require this, but catching it keeps the error handling explicit:

The catch block rethrows the IOException inside an UncheckedIOException, so this example ends with that error.

Explicit locking and unlocking

You can also acquire and release a lock explicitly with the lock and unlock methods, but functional locks are safer. If you lock explicitly, call unlock in a finally block so the lock is released even when an exception or error occurs. The doLocked and computeLocked methods do this for you.

The following example shows the correct way to use the shared lock with explicit locking:

Waiting for UpdateGraph changes with the exclusive lock

You can use the exclusive lock to wait for changes from the UpdateGraph.

An operation needs to wait when it depends on an UpdateGraph cycle that happens partway through it. For example, suppose your query flushes data to a DynamicTableWriter during initialization. The output table does not show that data until the UpdateGraph runs an update cycle.

The most common way to wait is await_update. Calling it on the output table acquires the exclusive lock and waits until that cycle has run, after which the flushed rows are visible. While it waits, it releases the lock, so the UpdateGraph keeps running cycles.

Note

Deephaven doesn't support upgrading from the shared lock to the exclusive lock. Attempting it throws an exception. Accordingly, you can't call await_update while your thread holds the shared lock.

When await_update isn't enough, an independent thread can hold the exclusive lock and wait on a condition that the UpdateGraph signals. Use this pattern when a thread that the UpdateGraph doesn't run must wait for a change or notification that the UpdateGraph propagates. Pair it with a custom listener. The pattern has three steps:

  1. Obtain a condition with updateGraph.exclusiveLock().newCondition(), where updateGraph is the UpdateGraph you capture from the table or execution context before you start the independent thread.
  2. In the independent thread, acquire the exclusive lock and await the condition. await releases the lock while the thread waits and reacquires it before returning.
  3. From code that the UpdateGraph runs, such as a listener's onUpdate method, call requestSignal. The UpdateGraph signals the condition while it holds the exclusive lock. A listener must not acquire either lock during an update cycle, and attempting it throws an exception. That's why the listener asks the UpdateGraph to signal the condition instead of signaling it directly.

The following Groovy example shows this pattern:

With the default number of update threads on a multicore machine, the UpdateGraph runs the listener on one of its update threads, so the Signaling condition... and Condition signaled. lines show a thread name such as PeriodicUpdateGraph-updateExecutor-1. The following output comes from a server configured with a single update thread, so those lines show PeriodicUpdateGraph.DEFAULT.refreshThread:

The output shows the progress of the two threads:

  1. myIndependentThread acquires the exclusive lock and waits for the condition. Waiting releases the lock until the condition is signaled.
  2. The UpdateGraph processes a timeTable tick, which triggers the listener's onUpdate method.
  3. onUpdate requests a signal for the condition.
  4. After the UpdateGraph finishes its cycle, myIndependentThread returns from await and resumes.

The time table adds its first row on the first update cycle after it is created, and later rows on five-second boundaries. The listener keeps running until you remove it from the table or it is garbage collected.

Snapshotting

A snapshot is a guaranteed-consistent result derived from data in Deephaven without necessarily acquiring a lock. Deephaven's snapshotting mechanism is a form of optimistic concurrency that helps to retrieve data safely, quickly, and without blocking the UpdateGraph.

ConstructSnapshot.callDataSnapshotFunction runs a snapshot. It attempts the operation without acquiring a lock, then verifies that the UpdateGraph did not change any data while the operation was running. If data may have changed while the operation was running, the snapshotting mechanism retries the operation.

If the snapshot cannot complete successfully after a limited number of attempts, or if a failed attempt runs longer than a configured duration, the snapshotting mechanism acquires the shared lock to perform the operation reliably. The Configuration section lists the properties that set these limits.

Lock-free attempts don't always happen. callDataSnapshotFunction skips them, and runs the snapshot function once, in two cases:

  • The calling thread already holds the shared or exclusive lock. Console commands hold the exclusive lock, so a snapshot that you take from the console runs this way.
  • The calling thread is an UpdateGraph thread, one of the threads that the UpdateGraph uses to run update cycles and listeners.

Obtaining snapshots

Obtaining a snapshot of tables or other data that the UpdateGraph updates takes two steps:

  1. Create a SnapshotControl object to control snapshot validation.
  2. Run the snapshot by passing the SnapshotControl and a SnapshotFunction, which holds the logic that must run on consistent data, to callDataSnapshotFunction.

Creating the SnapshotControl

To obtain snapshots, first use ConstructSnapshot.makeSnapshotControl to create a SnapshotControl object, which tells the query engine how to verify that the snapshot result is correct. The makeSnapshotControl parameters notificationAware, refreshing, and sources control this verification process.

Parameter NameDescription
notificationAwareWhether the snapshot result must capture every later notification from the sources. Set it to true when the snapshot initializes state that a listener then updates from those notifications. In most other cases, use false.
refreshingWhether the SnapshotControl needs to handle refreshing data. This must be true if any of the sources is refreshing. When it is false, the snapshot function runs once, with no lock and no consistency check.
sourcesA varargs parameter that must contain every source the snapshot function reads, including every table.

Running a snapshot

To run a snapshot, pass three arguments to callDataSnapshotFunction:

The snapshot function receives two arguments, usePrev and beforeClockValue. usePrev says which values to read:

  • true: the snapshot is running during an update cycle before the sources finish updating, so the function must read previous values, the values as of the start of the current cycle.
  • false: the function reads current values.

beforeClockValue is the logical clock value that the snapshotting mechanism read from the UpdateGraph before the attempt started. The logical clock changes when each update cycle starts and ends. The examples on this page don't use beforeClockValue.

Typically, a snapshot function retrieves data from several ColumnSources for one or more row keys, the keys that identify each row in a table. Consider the example below, which retrieves data at row key 0 from two tables:

For each attempt, callDataSnapshotFunction handles the result as follows:

Early stopping of inconsistent snapshots

A snapshot function that runs several operations can stop early when the data becomes inconsistent. Between operations, call ConstructSnapshot.concurrentAttemptInconsistent. It returns true if the sources or the UpdateGraph's logical clock have changed in a way that makes the current lock-free attempt inconsistent. It always returns false outside a lock-free attempt. When it returns true, return false from the snapshot function to end the attempt early. callDataSnapshotFunction then finds the data inconsistent and runs the function again.

In the following example, the snapshot function retrieves data from 100 columns in each of two tables and checks for consistency before it reads the second table:

Configuration

The following properties limit how many lock-free attempts a snapshot makes, and how long a failed attempt can run, before the snapshotting mechanism acquires the shared lock.

PropertyDefault ValueDescription
ConstructSnapshot.maxConcurrentAttempts2Maximum number of lock-free snapshot attempts before acquiring the shared lock.
ConstructSnapshot.maxConcurrentAttemptDurationMillis5000Maximum duration, in milliseconds, of a failed lock-free attempt before the snapshotting mechanism stops retrying and acquires the shared lock.