What is Barrage?

Barrage is Deephaven's streaming data protocol for efficient, real-time table transport between servers and clients. Barrage extends Apache Arrow Flight to support Deephaven's incremental update model, so subscribers receive only the data that has changed rather than the entire table on every update.

Why Barrage?

Two common ways to move real-time data between a server and a client have limitations:

• Polling: Repeatedly fetching entire tables wastes bandwidth and CPU cycles, especially for large tables with small changes.
• Arrow Flight alone: Apache Arrow Flight provides efficient data transport, but lacks native support for incrementally updating datasets. See Barrage vs. Arrow Flight for a feature-by-feature comparison.

Barrage solves these problems by:

• Streaming only the rows that have been added, removed, or modified.
• Supporting viewports so clients can subscribe to just the visible portion of a table.
• Batching updates to balance latency against network efficiency.

When to use Barrage

The following table matches common use cases to the Barrage feature that serves each one. Key concepts explains each feature.

Architecture overview

In the most common setup, one Deephaven server (the local server) captures a table that lives on another (the remote server):

1. Remote server hosts a table that a ticket refers to.
2. Barrage protocol transports the actual data using Arrow Flight with incremental update metadata.
3. Local server subscribes to or snapshots the table through a BarrageSession and receives a full local copy of the data. A subscription's copy stays synchronized with the source.

The local copy is an ordinary Deephaven table. Queries on the local server can join, filter, or aggregate it.

Key concepts

Shared tickets

A ticket is a reference to a table on a server. It points to the table but does not contain its data. Deephaven has several kinds of ticket, including:

• A scope ticket, which names a variable in the remote server's global scope. Any session, including the local server's Barrage session, can use it.
• An application ticket, which names a field that an Application Mode application exports on the remote server. Any session, including the local server's Barrage session, can use it.
• A shared ticket, which names a table that one session publishes for other sessions to use.
• An export ticket, which refers to a table that a session created and exported to its own client. An export ticket belongs to the client session that created it, so the local server's Barrage session can't use it.

To capture a scope variable or an application field, you can also resolve its URI with resolve. URI resolution works only against a remote server that allows anonymous authentication.

Other clients or servers can subscribe to or snapshot a shared ticket, which lets a session share a table that isn't in the global scope. The shared ticket stays valid only while the publishing session is open and the published table is alive.

The following example runs on a local Deephaven server that has the Python client, pydeephaven, installed (pip install pydeephaven). It uses the client to connect to a remote Deephaven server, create a ticking table there, and publish it to a shared ticket:

table_ref is a reference to the table on the remote server, not a Deephaven table on the local server. To get the data onto the local server, capture it through the shared ticket, as the next section shows.

Subscriptions vs. snapshots

Barrage supports two primary modes of retrieving data:

• Subscription: Opens a persistent connection that streams updates as the source table changes. The client receives an initial snapshot followed by incremental updates. Use this for ticking tables you want to monitor in real time.

• Snapshot: Retrieves a one-time, static copy of the table that doesn't update after Barrage delivers it. Use this for static tables or when you need a point-in-time capture.

This example continues the shared ticket example, which published a table to ticket on the remote server. To capture that table on the local server:

1. Call barrage_session to open a Barrage session to the remote server. It returns a BarrageSession.
2. Pass the shared ticket's bytes, ticket.bytes, to the session's subscribe or snapshot method.

Both methods return a Deephaven table on the local server:

The Barrage session stays open for more subscriptions and snapshots until you close it. See Capture Python client tables with Barrage for complete examples.

Like the shared ticket example, this code connects anonymously. See the note at the end of Shared tickets for how to connect to a server that uses pre-shared key authentication.

Viewports

A viewport defines a window over a table: a set of row positions, typically a contiguous range, and a subset of columns. Viewports are essential for interactive applications where users scroll through large tables. Rather than streaming millions of rows, the server sends only the data visible in the current view.

Deephaven's web UI manages viewports for you. When a user scrolls or resizes a table view, the UI updates its viewport subscription. Applications built on the JavaScript client set the viewport themselves with setViewport.

Update intervals and batching

Barrage batches table updates before sending them to subscribers. This batching reduces network overhead when tables update frequently. The update interval is configurable:

• Server default: Every subscription that doesn't request its own interval uses the server's default. Subscriptions made through barrage_session or resolve always use it. The update interval section of the Barrage performance guide gives the property that sets it.
• Per-subscription: The JavaScript client and the Java client can request a different interval for each subscription. The interval is fixed for the life of the subscription. To use a different interval, create a new subscription.

A shorter interval reduces latency but increases network traffic. A longer interval reduces traffic but introduces delay.

Barrage vs. Arrow Flight

Barrage uses the same columnar data format as Arrow Flight and adds the features this page has described for ticking tables:

Barrage builds on Arrow Flight, so a plain Arrow Flight client can fetch a static snapshot of a table with the Flight DoGet call, which downloads a stream of data for a ticket. The incremental update features require a Barrage-aware client.

Performance considerations

• Large initial snapshots: When a client subscribes to a large table, the initial snapshot can be memory-intensive. By default, the server breaks a large initial snapshot into smaller chunks and sends them over several update cycles. Use the subscription growth controls to tune the chunk size or turn off chunking.

• High-frequency updates: Tables that tick rapidly can generate significant network traffic. Consider increasing the server's default update interval or filtering the table before you subscribe to it.

• Wide tables: Publish or subscribe to a table that contains only the columns you need, for example one made with view before you share it. Fewer columns means less data to transfer.

• Unknown bottlenecks: Use the Barrage performance tables to track subscription health and identify bottlenecks.

Related documentation

• Interpret Barrage metrics
• Incremental update model
• Deephaven's table types
• Aggregations in Deephaven
• Designing the Deephaven Core API
• Capture Python client tables with Barrage
• Barrage metrics for performance monitoring
• Barrage schema annotation
• Arrow Flight and Deephaven
• Use URIs to share tables
• Application Mode
• Python client quickstart
• Enable anonymous authentication
• Configure and use pre-shared key authentication
• Use the JS API
• Extract data with a Java client
• Filter table data
• Exact and relational joins
• barrage_session
• subscribe
• snapshot
• close
• URI
• view
• where
• JS API concepts
• Barrage protocol documentation
• deephaven.barrage Pydoc
• pydeephaven.session.Session Pydoc
• pydeephaven.ticket Pydoc
• BarrageSubscription Javadoc