Have you ever sent a URL to a friend or family member so they can read, watch, or see something you enjoy on the web? I've done it more times than I could probably count. I've also had times where I wanted to share my Deephaven tables without forcing others to run my query or understand how I got my results. So why can't I share those like I do the URLs with my friends? As it turns out, I can.
The Deephaven URI
We at Deephaven are excited to announce the Deephaven URI - a way to share Deephaven objects through a link. Open tables from other Deephaven instances with a single line of code. It's as easy as loading a URL in your web browser.
Deephaven's URIs standardize the URI (Uniform Resource Identifier) format for Deephaven targets. Since a URL is a type of URI, that means that sharing tables with others will feel remarkably familiar - like sharing a webpage with someone.
Why is this exciting? Because now, instead of having to share code to share tables, you can now just share a link. Or, if the recipient knows your host, then they only need a table name. Don't believe me? Let's try it out below.
Use cases
Each subsection below explores a use case for sharing across workers, networks, and publicly.
Worker-to-worker sharing locally
Deephaven tables can be shared in a worker-to-worker context on the same machine. Why would anyone want to share results on the same machine? Outside of an exemplary use case, this is particularly powerful on shared workstations, as often seen in the business world.
To demonstrate sharing on the same machine, I'll create a docker-compose.yml file that creates two Docker containers, each with Deephaven. The first, table-producer, runs on port 10000 and does what its name suggests: produce a table. It's a real-time table with some ticking data. The second, table-consumer, runs on port 9999 and is also aptly named because it will get the table being created on table-producer with a URI.
Here is my docker-compose.yml file:
version: '3'
services:
table-producer:
image: ghcr.io/deephaven/server:0.17.0
ports:
- '10000:10000'
table-consumer:
image: ghcr.io/deephaven/server:0.17.0
ports:
- '9999:10000'
After a docker compose pull and docker compose up --build -d, I've got two instances of Deephaven up and running.
In my table-producer instance, I want to create some real-time data. Let's do just that.
from deephaven import time_table
real_time_data = time_table("00:00:01").update(["X = 0.1 * i", "Y = sin(X)"])
Piece of cake. Now, I head over to my table-consumer container and resolve it. I need three things: the container name, port, and the table name. Thankfully, I know all of these.
from deephaven import uri
real_time_data_resolved = uri.resolve("dh+plain://table-producer:10000/scope/real_time_data")
Oh, and if you want to use Groovy rather than Python, it's just as easy.
import static io.deephaven.uri.ResolveTools.resolve
real_time_data_resolved = resolve("dh+plain://table-producer:10000/scope/real_time_data")
And that's it. It's all I needed to do in order to share a table from worker to worker.
Worker-to-worker sharing across a network
Since this use case has nearly identical syntax as the local case, I won't show an example. Rather, I'll highlight the two differences.
In the local example, I resolved the table using the Docker container name. In the case of sharing across a network, I need the IP/hostname instead. So, to build on the previous example. let's say I'm running on the private IP 192.168.10.1. My colleague is on a different machine. I call up my colleague, tell him I've got real_time_table ready for him, and all he has to do is:
from deephaven.uri import resolve
real_time_data_resolved = resolve("dh+plain://192.168.10.1:10000/scope/real_time_table")
Share publicly
All of the same principles apply here that applied when sharing across a private network. Any consumer could grab a Deephaven table with an IP or hostname, port, and table name. So, if Wikipedia ever decides to host a Deephaven table, go grab it in a single line of code with a URI!
Deephaven concepts for publishing
Deephaven's table update model, powered by Barrage, powers the publishing model for real-time streams. Barrage allows Deephaven to deliver incremental updates to downstream users and consumers in an efficient way.
Deephaven's URI package benefits from the update model. It allows you to inherit results of complex queries without needing to know the details. Tables, both static and real-time, can easily be consumed by Deephaven instances external to your own. It doesn't matter if that instance on the same machine, the same network, or a public-facing website. The process for using them is intuitive all the same.
Share your thoughts with us
Have questions for us? Or do you have a use case you'd like to implement? We'd love to hear about them and help you get them up and running! Reach out to us on Slack.