Liveness Scopes

This guide discusses liveness scopes. It covers why queries can benefit from a liveness scope, how liveness scopes work, and how to create and use one.

Why use a liveness scope?

Deephaven's engine runs in Java, where the JVM handles garbage collection (GC). Users have little control over when GC runs, because the JVM decides when to collect based on memory demand.

A liveness scope lets your code choose when to release the query engine objects it no longer needs, instead of waiting for the JVM. Releasing a scope cleans up:

  • Tables and plots.
  • Other query engine objects, such as the listeners that connect tables in the update graph.

Without a scope of their own, queries rely on JVM garbage collection, which is fine in most cases. A scope helps when a query repeatedly creates ticking tables that it needs only for a while, such as a function that rebuilds its tables each time its parameters change. Releasing the scope cleans up the old tables right away. Otherwise they keep processing updates until garbage collection runs.

How liveness scopes work

Deephaven uses reference counting to decide when to clean up tables, plots, and many other query engine objects. Each object counts the references to it, and Deephaven cleans the object up as soon as the count drops to zero. An object tracked this way is a liveness referent. A referent is live while its count is above zero. A liveness scope is an object that holds references to referents. Deephaven tracks references as follows:

  • Open liveness scopes form a stack on each thread. When a query creates a referent, the scope at the top of the stack manages it, which adds a reference. While the Python console runs your code, the console's own liveness scope is open, so by default it manages the tables and other referents your script creates.
  • A refreshing (ticking) table holds a reference to each refreshing parent table it depends on, so a parent stays live as long as one of its children does.
  • When a referent's reference count drops to zero, Deephaven releases it immediately. A released refreshing table stops processing updates and drops its references to its parents, which can in turn be released.
  • Releasing a scope drops its references to everything it manages. Referents that nothing else references are released. The rest, such as a result you preserve in the enclosing scope (the scope below it on the stack), stay live.

Reference counting does not replace GC. A variable that refers to a table is not one of these counted references. Suppose no variable refers to a refreshing table, no live table depends on it, and no scope that manages it has been released. The table keeps processing updates until the JVM garbage collector reclaims it.

How to create and use a liveness scope

Creating a liveness scope is easy. Call the liveness_scope function, or create a scope directly from the LivenessScope class. Neither takes arguments.

  • Use the liveness_scope function only in a with block or as a function decorator.
  • The LivenessScope class gives you finer control, because you can open a scope more than once and release the objects it manages when you choose.

Use the function when a scope covers one block or one function call. Use the class when you need to release the scope later, from code outside the block that created it.

The liveness_scope function

Entering a with liveness_scope() block, or calling a function decorated with @liveness_scope(), creates a SimpleLivenessScope, which can only be opened once. While it is the innermost open scope, the scope manages the query engine objects your code creates, and it drops its references to them when the block or decorated function exits. Use preserve to keep a result live in the enclosing scope.

The following example builds the same ticking query in two functions:

  • get_latest preserves the table it returns, so latest_values keeps ticking after the with block exits. Its parent, the ticking_table in get_latest, also stays live, because the preserved table depends on it.
  • get_values preserves nothing, so its scope releases every table it creates when the function returns. Only the NumPy array, which is a copy of the data, remains.

The LivenessScope class

Use the LivenessScope class when the scope must outlive one with block and you want to choose when to release it.

In the following example, each call to make_table_and_scope creates a filtered ticking table inside a new scope and returns both. The scope stays unreleased after the with block, so the table keeps ticking. Releasing the scope stops the table, even though t1 still refers to it. Two things keep it updating after the release:

  • Another scope, or a refreshing child table, that holds a reference to it.
  • The UI, while the table is open there.

Multiple liveness scopes

Liveness scopes can be nested. Because open scopes form a stack, the innermost open scope manages new objects. When its with block exits, the scope that encloses it manages new objects again.

In the following example, the inner scope manages ticks and evens. Inside the inner block, preserve hands ticks to outer_scope. Inside the outer block, preserve hands it on to the console's scope.

evens is released when the inner block exits, so it stops updating. ticks keeps ticking, because each scope preserved it before it closed.

Scope methods

Scopes created by both the liveness_scope function and the LivenessScope class have these methods:

  • manage(referent) adds a reference from this scope to the object, so the object stays live until the scope is released.
  • preserve(referent) keeps the object live in the enclosing scope. The scope must be open and at the top of the stack when you call it.
  • unmanage(referent) drops one reference from this scope to the object.

Scopes created directly from the LivenessScope class also have these methods:

  • open opens the scope for use in a with statement.
  • release closes the scope and drops its references to everything it manages.

Example: release a query's intermediate tables

Queries often create intermediate tables that you don't need to refer to once the query has run. Without a liveness scope of their own, those tables stay managed by the console's scope. Let's look at the problem first, and then solve it with a scope.

This query uses read_csv, merge, and tree to create a simple tree table grouped by Instrument. Two tables remain in the console: crypto and combo_tree.

The above example works, but the query creates many objects that the console doesn't need to refer to by name once the query finishes: data, combo, and the unnamed table that each chained operation produces. Setting data and combo to None removes their names from the console, but it doesn't release the tables.

The following example runs the same query inside a LivenessScope. Inside the scope, preserve keeps crypto and combo_tree live in the console's scope. Calling release then releases the intermediate tables, because no other scope or table manages them. The last two lines remove the released tables' names from the console.

The release succeeds, but it frees no memory here. Every intermediate table is an ancestor of combo_tree, which still reads their data. Because read_csv reads the file once, these tables are static and have no updates to stop. If they were refreshing, combo_tree would keep them live and updating. A scope frees resources only for tables that nothing you keep depends on.

The next example shows that case with ticking data. latest is the result you keep, and it reads from trades. every_third also reads from trades, but nothing you keep reads from every_third.

After release, every_third stops updating, because nothing you kept depends on it. latest and its parent trades keep ticking, because the preserved latest depends on trades. As in the LivenessScope class example, a table that is open in the UI keeps updating until you close it.