Local development with Deephaven libraries

This guide explains how to build Python projects that use Deephaven tables locally. This is useful for creating utilities, custom functions, or data pipelines that work with Deephaven tables outside of the Deephaven IDE.

Set up your project

Install deephaven-server to use Deephaven tables in your Python project:

pip install deephaven-server

Note: deephaven-server requires Java 17+ and sets up an embedded Deephaven server. Set your JAVA_HOME environment variable before running.

Use case	Package
Connect to a remote Deephaven server	`pydeephaven`
Read/write Parquet files	`deephaven.parquet` module (included with `deephaven-server`)
NumPy integration	`numpy` (included with `deephaven-server`)
Pandas integration	`pandas` (included with `deephaven-server`)

Local unit testing

To use Deephaven table operations in unit tests, start an embedded server before importing Deephaven modules.

pytest setup

Install pytest alongside deephaven-server:

pip install pytest

Create a conftest.py file in your test directory to start the server once for all tests:

# conftest.py
from deephaven_server import Server

# Start the server at module load time.
# pytest loads conftest.py before collecting test modules, so this runs
# before any test file imports deephaven.
# Port 0 lets the OS pick a free port, so the tests don't collide with a
# Deephaven server that is already running.
_server = Server(port=0, jvm_args=["-Xmx2g"])
_server.start()

Note: The server must start before any deephaven imports. Since pytest loads conftest.py before collecting test modules, starting the server at module level ensures the JVM is ready when test files import deephaven.

Example tests

# test_my_utils.py
from deephaven import empty_table, new_table
from deephaven.column import int_col, string_col


def test_table_creation():
    """Test that we can create a table."""
    t = empty_table(10).update(["X = i", "Y = X * 2"])

    assert t.size == 10
    assert "X" in [col.name for col in t.columns]
    assert "Y" in [col.name for col in t.columns]


def test_table_operations():
    """Test filtering and aggregation."""
    t = new_table(
        [
            string_col("Category", ["A", "B", "A", "B", "A"]),
            int_col("Value", [10, 20, 30, 40, 50]),
        ]
    )

    # Filter
    filtered = t.where("Category = `A`")
    assert filtered.size == 3

    # Aggregate
    summed = t.sum_by("Category")
    assert summed.size == 2

Run tests with:

pytest tests/ -v

Testing ticking tables

When testing with time_table or other ticking tables, use await_update to wait for updates:

from deephaven import time_table


def test_ticking_table():
    """Test a ticking table."""
    t = time_table("PT1S").update("X = ii")

    # Wait for the table to have at least 3 rows, up to 5 seconds total
    attempts = 0
    while t.size < 3 and attempts < 5:
        t.await_update(1000)  # Wait up to 1 second
        attempts += 1

    assert t.size >= 3

Testing with a remote server

If your code connects to a remote Deephaven server, use pydeephaven:

pip install pydeephaven

See Python Client Quickstart for full setup instructions.

# test_client.py
import pytest
from pydeephaven import Session


@pytest.fixture(scope="module")
def session():
    """Connect to a running Deephaven server."""
    # Deephaven servers require PSK authentication by default; see the PSK
    # printed to the server's log on startup.
    session = Session(
        host="localhost",
        port=10000,
        auth_type="io.deephaven.authentication.psk.PskAuthenticationHandler",
        auth_token="YOUR_PASSWORD_HERE",
    )
    yield session
    session.close()


def test_fetch_table(session):
    """Test fetching a table from the server."""
    # Assumes 'my_table' exists on the server
    table = session.open_table("my_table")
    assert table.size > 0

Reading test data

Load test data from CSV or Parquet files:

from deephaven import read_csv
from deephaven.parquet import read


def test_csv_data():
    """Test with CSV data."""
    t = read_csv("tests/data/test_data.csv")
    assert t.size > 0


def test_parquet_data():
    """Test with Parquet data."""
    t = read("tests/data/test_data.parquet")
    assert t.size > 0

See read_csv and read for more options.

Project structure

A typical project structure:

my_project/
├── src/
│   └── my_utils.py
├── tests/
│   ├── conftest.py      # Server setup
│   ├── test_my_utils.py
│   └── data/
│       ├── test_data.csv
│       └── test_data.parquet
├── pyproject.toml
└── README.md

Example `pyproject.toml`

[project]
name = "my-deephaven-project"
version = "0.1.0"
requires-python = ">=3.9"
dependencies = [
  "deephaven-server>=42.7",
]

[project.optional-dependencies]
test = [
  "pytest>=7.0",
]

[tool.pytest.ini_options]
testpaths = ["tests"]