Use URIs to share tables

This guide shows you how to use Deephaven URIs and io.deephaven.uri.ResolveTools to share tables across server instances and networks.

A URI, short for Uniform Resource Identifier, is a sequence of characters that identifies a resource on the web. Think of a URI as a generalization of a URL. A Deephaven URI identifies a table on a server instance. By linking to a URI, you can work with live tables from other Deephaven server instances without rebuilding the data pipelines or queries that created them.

Why use URIs?

Deephaven URIs provide several key benefits:

• Canonicalized resource identification: Access query-scope tables and application fields through a standardized string format that works across server instances.
• Simplified data sharing: Share tables between Deephaven instances without duplicating the queries that create them.
• Distributed computing: Build systems that spread processing across multiple Deephaven servers.
• Real-time access: Access live, updating tables from remote sources that reflect the latest data.
• Cross-language compatibility: A Python server can resolve tables from a Groovy server, and the reverse. See the Python version of this guide.
• Environmental isolation: Access data across different containers, servers, or networks.

By using URIs, you let others work with your live tables without rebuilding your data pipeline, understanding your query logic, or keeping a copy of the data up to date themselves. This is particularly valuable in collaborative environments and distributed systems.

Syntax

Like a URL such as https://deephaven.io/core/docs, a Deephaven URI has a scheme (https), an authority (the host, deephaven.io), and a path (/core/docs). In a Deephaven URI, the path is the scope followed by the resource name, and the authority can include a port.

Deephaven URI structure

Deephaven URIs use this syntax:

The components are:

• dh+plain or dh is the scheme, which identifies the protocol for accessing Deephaven resources.
  • dh:// indicates a secure connection (TLS/SSL).
  • dh+plain:// indicates an insecure connection (no encryption).
• <host> is the host, which is either:
  • A Docker container name or Docker Compose service name (for container-to-container communication within the same Docker network).
  • A hostname or IP address (for network communication).
• <port> is optional. Without it, the client connects to the scheme's default port: 10000 for dh+plain://, 443 for dh://. Include the port when the remote Deephaven instance listens on a different port, for example dh+plain://hostname:9876/scope/tableName.
• <scope> identifies the namespace where the resource exists, usually scope or app/<app_name>/field. See Resource scopes and paths.
• <resource_name> is the exact name of the table you want to access. Resolving a remote dh or dh+plain URI returns a table, so the name must refer to a table.

Resource scopes and paths

A scope in a Deephaven URI is a namespace that identifies where a resource exists within a Deephaven server instance. Scopes provide:

• Separate namespaces: A table named trades in the query scope is separate from a field named trades in an application scope.
• Clear resource organization: You know immediately whether a resource comes from interactive work or a pre-built application.

This guide covers the two scopes you use most. The URI for each one has this form:

resolve also reads CSV files, from a local path (csv:///data/file.csv) or over HTTP(S) (csv+https://host/path/file.csv), and local Parquet files or directories (parquet:///data/file.parquet).

Query scope (scope)

The query scope holds the variables that a session creates. A table or other object goes into the query scope when you:

• Create it in the Deephaven IDE console.
• Create it in a script that a client runs on the server.
• Bind a client table to a name, for example with the Python client's Session.bind_table.

Application scope (app/<app_name>/field)

The application scope contains fields exported by Application Mode applications, which are available as soon as the server starts. The <app_name> in the URI is the application's ID: the id in the .app file for a script application, or the ID that the application class sets for a static or dynamic application. For example, a field named marketData in an application with id=tradingApp is available at dh://hostname/app/tradingApp/field/marketData.

Resolve a URI

To access a table via its URI, use the resolve method from io.deephaven.uri.ResolveTools:

For a dh:// or dh+plain:// URI, resolve connects to the specified Deephaven instance, creates a Barrage subscription to the remote table, and waits for the initial snapshot of its data before it returns. The table it returns is always a refreshing table, which means it can receive updates, even when the remote table is static. It receives the remote table's later changes in update cycles.

To wait for the next change from a ticking remote table, use awaitUpdate. Pass a timeout in case the remote table is static, because a static table never sends another update:

A resolved table rides out a brief network interruption: when the connection returns, the subscription resumes and catches up on the changes it missed. If the producer restarts, or the connection stays down long enough for the producer to expire the consumer's session, the resolved table fails and stops updating. Calling resolve again with the same URI doesn't help, because the consumer reuses the session it cached for that server, and the producer no longer accepts it. To get the table back, restart the consumer and resolve the URI again.

To work with a fixed copy of a resolved table, for example when every operation in an analysis must see the same data, use snapshot. The copy is static and holds the table state from the moment you take it, while the resolved table keeps receiving updates.

Let's explore this with two complete examples: sharing a table between two Docker containers on one machine, and sharing a table across a network.

Share tables locally

This first example uses two Docker containers that run Deephaven with Groovy, each exposed on a different host port.

Docker Compose

Spinning up multiple Deephaven instances from Docker takes only a short Compose file. This example creates two containers. Both listen on port 10000 inside their containers. The host reaches table-producer on port 10000 and table-consumer on port 9999. The docker-compose.yml file looks like this:

After a docker compose pull and docker compose up -d, both instances are up and running.

Without Docker Compose

If you're not using Docker Compose, you can create a Docker network manually to enable container name resolution:

Create a table on the producer

Open the table-producer IDE at http://localhost:10000/ide and create a real-time table with timeTable:

Resolve the table on the consumer

Open the table-consumer IDE at http://localhost:9999/ide. To get the producer's table, the consumer needs its URI. The URI consists of the scheme, the producer's Docker container name (its service name when you use Docker Compose), the scope namespace, and the table name. In this example, the URI is dh+plain://table-producer/scope/myTable.

In the table-consumer IDE, resolvedTable now ticks as myTable on the producer adds rows.

Share tables across a network

You can also share tables across networks, public or private. This works the same way as sharing tables locally, where the consumer passes the producer's URI to resolve. In the URI, replace the container name with the hostname or IP address of the instance that produces the table. Add the port only when that instance doesn't listen on the scheme's default port, as described in Deephaven URI structure.

Create a table on your colleague's server

Suppose you're on a private network, and your colleague runs Deephaven on port 9876 on a machine with IP 192.168.5.1. Their server must allow anonymous authentication, because your server connects to it anonymously when it resolves the URI. For example, they can start it with -DAuthHandlers=io.deephaven.auth.AnonymousAuthenticationHandler, as in the Docker Compose example.

On that server, your colleague creates a table:

Resolve your colleague's table

Once again, the URI consists of the scheme, the colleague's IP address and port, the scope namespace, and the table name:

If you know the hostname of your colleague's machine, you can use it in place of the IP address. If the machine is public, use its public hostname or IP address. Keep in mind that anyone who can reach a server with anonymous authentication can connect to it, so expose such a server only on a network you trust, and use dh:// so that the connection is encrypted.

Performance considerations

When you share tables across instances with URIs, particularly over networks, consider these performance factors:

Network impact

• Latency: Table access over a network introduces latency that varies based on network conditions. When an operation needs low latency, run the instances on the same machine or network.
• Bandwidth: Deephaven's Barrage protocol transmits only changes after the initial snapshot rather than full table refreshes, so bandwidth use depends mostly on the table characteristics below.
• Connection reliability: A resolved table recovers from a brief network interruption, but not from a producer restart or a long outage, which require restarting the consumer (see Resolve a URI). Co-locate the instances, or resolve over a reliable network link, when the consumer can't tolerate an interrupted subscription.

Table characteristics

• Full-table subscription: resolve subscribes to every row and column of the remote table, and the consumer keeps its own copy of the table in memory.
• Initial snapshot: When you first resolve a URI, the remote server sends a snapshot of the current table state. Larger tables require more resources for this initial transfer.
• Update frequency: Tables that tick often send more data over the network after the initial snapshot, so they use more bandwidth.
• Column types: Columns that hold large values, such as long strings, arrays, or nested structures, send more data during the initial snapshot and each later update.

Optimization strategies

• Only share what's needed: Filtering a resolved table on the consumer doesn't reduce what crosses the network, so share only the data a downstream consumer needs: apply filters at the source, select only the necessary columns, and pre-aggregate large datasets to reduce the volume of transferred data.
• Avoid repeated URI resolution: Store resolved table references in variables rather than calling resolve multiple times for the same URI. resolve reuses an existing connection when a URI names the same scheme, host, and port as an earlier one, but each call creates a new Barrage subscription to the remote table.

Related documentation

• emptyTable
• timeTable
• update
• resolve
• Enterprise URIs
• Capture remote tables with Barrage
• Application Mode
• URI cheat sheet
• io.deephaven.uri Javadoc