Configure the Deephaven Docker application

This guide covers best practices for configuring the Deephaven Docker application. Deephaven can be run from Docker alone or with Docker Compose. The latter is recommended for customized applications, as it provides a simpler mechanism for orchestrating complex applications with multiple services.

For lower-level details, such as building your own Docker images from scratch, see the production application guide. This guide assumes some knowledge about Docker.

This guide shows how to run the Deephaven Docker image with custom JVM options, configuration, and ports. It covers the following tasks:

  • Running the image with Docker
  • Setting JVM options
  • Using a configuration file
  • Changing the port
  • Customizing the image
  • Adding Python packages
  • Running the image with Docker Compose
  • Importing custom JARs
  • Debugging a running container

The available images section comes right after the configuration mechanisms. The sections on image contents and versioning come after the task sections.

The image build logic is in the deephaven-server-docker repository.

Configuration mechanisms

The image accepts configuration through these main mechanisms:

MechanismWhere you set itUse it for
START_OPTSEnvironment variableAdding JVM arguments and system properties to the defaults.
JAVA_OPTSEnvironment variableReplacing the default JVM options file, such as to change the garbage collector.
Configuration file/opt/deephaven/config/deephaven.prop, mounted with -v or copied in a DockerfileSetting many Deephaven properties.
Docker Compose environment and portsCompose file or shellSetting values for one service in a Compose application.

Available images

Standard images

  • ghcr.io/deephaven/server: Deephaven's server-side Python API with jpy, deephaven-plugin, numpy, pandas, pyarrow, and the deephaven.ui, Deephaven Express (deephaven-plugin-plotly-express), and Matplotlib plugins.
  • ghcr.io/deephaven/server-ui: The same image as ghcr.io/deephaven/server, under a separate name.

The edge image includes Numba. The images for releases 41.3 through 42.x don't, so install Numba yourself if you use one of those releases.

For the Groovy image, ghcr.io/deephaven/server-slim, see the Groovy version of this guide.

Extended Python images

These images include the core Python API (deephaven-core) plus the listed libraries. They do not include the plugins bundled in ghcr.io/deephaven/server, such as deephaven.ui, Deephaven Express (deephaven-plugin-plotly-express), and Matplotlib.

  • ghcr.io/deephaven/server-all-ai: Deephaven's server-side API with NLTK, TensorFlow, PyTorch, and scikit-learn.
  • ghcr.io/deephaven/server-nltk: Deephaven's server-side API with NLTK.
  • ghcr.io/deephaven/server-pytorch: Deephaven's server-side API with PyTorch and TensorBoard.
  • ghcr.io/deephaven/server-sklearn: Deephaven's server-side API with scikit-learn.
  • ghcr.io/deephaven/server-tensorflow: Deephaven's server-side API with TensorFlow and TensorBoard.

Run with Docker

Note

Deephaven recommends running with the latest image to stay up-to-date with the latest features, bug fixes, and patches. See Versioning.

The quickest way to get up and running with the Deephaven Docker application is to run it with the default configuration:

Set JVM options

To add additional JVM arguments to the default configuration, set START_OPTS. The following START_OPTS tells the JVM to start with 4GB of heap memory, but to be able to use up to 6GB if necessary:

The defaults already select the G1 garbage collector. The following START_OPTS tries to select the Z Garbage Collector as well, so the JVM fails to start:

To use the Z Garbage Collector, set JAVA_OPTS instead. Setting JAVA_OPTS replaces the image's VM options file, /opt/deephaven/server-jetty/etc/dh-default.vmoptions. That file sets these defaults:

  • The G1 garbage collector
  • The pause-time target
  • String deduplication
  • -XX:GCLockerRetryAllocationCount=128

Include any of those settings you still want:

Caution

JAVA_OPTS replaces the image's VM options file, so any default option you leave out is lost. START_OPTS keeps the defaults, so use it unless you need to change them. Take care when overriding default VM options.

START_OPTS takes JVM arguments, including system properties, which are JVM -D settings. If you set many Deephaven configuration values this way, use a configuration file instead.

Use a configuration file

Mount your file over the image's copy with -v. This bind mount maps a host file or folder into the container. Start the file with includefiles=dh-defaults.prop so it keeps the default properties, and then add your own properties:

Change the port

The images expose port 10000 by default. To change the host port, use a port mapping such as -p 10001:10000.

To serve on a different port inside the container, set the http.port property in a configuration file. The image's health check probes port 10000, so the container reports as unhealthy after you change http.port.

With Docker Compose, set the DEEPHAVEN_PORT shell variable to change the host port. Compose substitutes it into the ports entry of the Compose file:

To change the default port in the Compose file instead, change the fallback value in the ports entry:

Customize the image

You can build your own image that includes your configuration:

The COPY instruction replaces the image's configuration file. Start your deephaven.prop with includefiles=dh-defaults.prop, as described in Use a configuration file, and then add your own properties.

EXTRA_CLASSPATH lists the directories whose JARs join the classpath. /apps/libs/* is the image default (see Import custom JARs). The second entry, /opt/more_java_libs, is an example. Replace it with the directory where your Dockerfile places your JARs.

Add Python packages

Deephaven's Python Docker images include a virtual environment in /opt/deephaven/venv and set VIRTUAL_ENV=/opt/deephaven/venv. Install additional Python packages in your own image:

Run with Docker Compose

Deephaven recommends running with Docker Compose, as it makes it easier to run Deephaven with additional services, such as:

  • Kafka brokers
  • Backend databases
  • Frontend applications

You write Docker Compose files in YAML. The most basic Docker Compose file for Deephaven with Python is:

VERSION sets the image tag. It defaults to latest. Set it in the shell, for example with export VERSION=42.7, to pin a version. See Versioning.

To change the port, see Change the port.

To build the image from a Dockerfile in the same directory, replace the image line with build: . in the example above.

To run a Docker Compose application, use:

Deephaven offers a variety of pre-built Docker Compose files that can be used to quickly get up and running with Deephaven. They cover both the Python and Groovy APIs. They also add services, such as example data and Redpanda, and extended Python images.

Import custom JARs

You can make your own Java classes (and third-party libraries) available to queries by placing JARs under /apps/libs. The images add that directory to the classpath (see Image contents and startup settings).

Option A: bind mount
Option B: Docker Compose

In a Docker Compose file, mount the folder in the service definition:

You can run queries that use your custom JARs. For example, if you have a class org.example.MathFns with a static method square(long x), you can run the following query:

Debug a running container

To debug a container started with docker run, use the docker exec command. The following command creates an executable bash shell in a running Docker container called deephaven-application. Start the container with --name deephaven-application to use this name, or replace the name with the container name or ID that docker ps shows:

With Docker Compose, run docker compose exec deephaven /bin/bash.

Image contents and startup settings

Deephaven's server Docker image is based on the production application, which the image unpacks into the /opt/deephaven directory. The images have two volumes:

  • /data
  • /cache

The images expose port 10000 by default. For changing the port, see Change the port.

A configuration file exists at /opt/deephaven/config/deephaven.prop. It includes the default configuration file dh-defaults.prop and sets deephaven.console.type to python.

Add additional Java JARs to /apps/libs, as described in Import custom JARs.

By default, Deephaven's Docker images set the following environment variables:

  • DEEPHAVEN_DATA_DIR=/data
  • DEEPHAVEN_CACHE_DIR=/cache
  • DEEPHAVEN_CONFIG_DIR=/opt/deephaven/config
  • EXTRA_CLASSPATH=/apps/libs/*

The first three are bootstrap configuration parameters, which the server reads at startup before it reads any configuration file. The start script reads EXTRA_CLASSPATH.

Set JVM options describes the JVM arguments and how to change them.

Versioning

The examples on this page use the latest version for Docker images. Here is a list of available versions:

  • latest: The latest release of Deephaven.
  • edge: Server code merged to the main branch of deephaven-core at the time the image was built, which may include breaking changes.
  • 42.7 or 41.11: A specific version number.

Older versions of Deephaven are likely to be incompatible with code found in documentation.

The edge image is rebuilt nightly, and whenever the main branch of the deephaven-server-docker repository changes. It is not rebuilt on every push to deephaven-core main. Use it only to try an unreleased feature or develop against the latest code.