Configure the Deephaven Docker application

This guide covers best practices for configuring the Deephaven Docker application. Deephaven can be run from Docker alone or with Docker Compose. The latter is recommended for customized applications, as it provides a simpler mechanism for orchestrating complex applications with multiple services.

If you are interested in lower-level details, which may be necessary if you are interested in building your own Docker images from scratch, see the production application guide.

Construction

Deephaven's server Docker image is based on the production application, which is unpacked into the /opt/deephaven directory. The images have two volumes:

• /data
• /cache

The images expose port 10000 to users by default. This value can be changed. Additionally, a configuration file exists at /opt/deephaven/config/deephaven.prop, which contains the default configuration values for the Deephaven server. Additional Java JARs can be included in /apps/libs.

Deephaven's Docker images, by default, set the following bootstrap environment configuration values:

• DEEPHAVEN_DATA_DIR=/data
• DEEPHAVEN_CACHE_DIR=/cache
• DEEPHAVEN_CONFIG_DIR=/opt/deephaven/config
• EXTRA_CLASSPATH=/apps/libs/*

Deephaven Docker images can set two environment variables to override or add additional JVM parameters. The semantics for these options remain the same as for the production application:

• JAVA_OPTS: Custom JVM properties that override default values.
• START_OPTS: Additional JVM properties that are appended to the default values.

caution

START_OPTS is recommended over JAVA_OPTS because it is safer. Take care when overriding default properties.

Deephaven's Python Docker images further have a virtual environment in /opt/deephaven/venv and set the environment value VIRTUAL_ENV=/opt/deephaven/venv.

For more information on the exact construction of the images, the image building logic is located in the deephaven-server-docker GitHub repository.

Versioning

Everything below will use the latest version for Docker images. Deephaven recommends staying on the latest version to get the latest features, bug fixes, and patches. Here is a list of available versions:

• latest: The latest release of Deephaven.
• edge: Everything new in the server up until midnight of the previous day. This version may contain features not yet released in latest that include breaking changes. This version is only recommended if you wish to try an unreleased feature or develop against the latest code.
• Specific version number: You can also uses a specific version, such as 0.35.0, 0.34.2, and more. Older versions of Deephaven are likely to be incompatible with code found in documentation.

Run with Docker

note

Deephaven recommends running with the latest image to stay up-to-date with the latest features, bug fixes, and patches.

The quickest way to get up and running with the Deephaven Docker application is to run it with the default configuration:

docker run --rm -p 10000:10000 ghcr.io/deephaven/server:latest

To add additional JVM arguments to the default configuration, set START_OPTS. The following START_OPTS tells the JVM to start with 4GB of heap memory, but to be able to use up to 6GB if necessary:

docker run --rm --env START_OPTS="-Xms4g -Xmx6g" ghcr.io/deephaven/server:latest

It's possible that some options conflict with the Deephaven recommendations. The following START_OPTS tells the JVM to use the Z Garbage Collector, but this conflicts with Deephaven's recommendations:

$ docker run --rm --env "START_OPTS=-XX:+UseZGC" ghcr.io/deephaven/server:latest
Error occurred during initialization of VM
Multiple garbage collectors selected

In this case, use JAVA_OPTS to override the default value:

docker run --rm --env "JAVA_OPTS=-XX:+UseZGC" ghcr.io/deephaven/server:latest

START_OPTS is a convenient way to set system properties. If you find yourself (ab)using system properties to set a lot of Deephaven configuration file values, consider instead overriding the default properties with a custom configuration file:

docker run --rm -v /path/to/deephaven.prop:/opt/deephaven/config/deephaven.prop ghcr.io/deephaven/server:latest

You can also build your own image with all of the necessary configuration options baked in:

Dockerfile

FROM ghcr.io/deephaven/server:latest
COPY deephaven.prop /opt/deephaven/config/deephaven.prop
ENV START_OPTS="-Xmx2g"
ENV EXTRA_CLASSPATH="/apps/libs/*:/opt/more_java_libs/*"

# Build the image from the Dockerfile
docker build -t my-deephaven-image .
# Run the image
docker run --rm -p 10000:10000 my-deephaven-image

You can also install additional Python dependencies into your own image if desired:

Dockerfile

FROM ghcr.io/deephaven/server:latest
RUN pip install some-awesome-package

Standard images

• ghcr.io/deephaven/server: Deephaven's server-side Python API. Includes jpy, deephaven-plugin, numpy, pandas, and numba.
• ghcr.io/deephaven/server-slim: Deephaven's server-side Groovy API.

Extended Python images

• ghcr.io/deephaven/server-ui: Deephaven's server-side API with deephaven.ui and other plugins pre-installed.
• ghcr.io/deephaven/server-all-ai: Deephaven's server-side API with NLTK, Tensorflow, PyTorch, and SciKit-Learn.
• ghcr.io/deephaven/server-nltk: Deephaven's server-side API with NLTK.
• ghcr.io/deephaven/server-pytorch: Deephaven's server-side API with PyTorch.
• ghcr.io/deephaven/server-sklearn: Deephaven's server-side API with SciKit-Learn.
• ghcr.io/deephaven/server-tensorflow: Deephaven's server-side API with Tensorflow.

Debugging

To debug a running Docker container, use the docker exec command. The following command creates an executable bash shell in a running Docker container called deephaven-application:

docker exec -it deephaven-application /bin/bash

Run with Docker Compose

Deephaven recommends running with Docker Compose, as it makes it easier to run Deephaven with additional services, such as:

• Kafka brokers.
• Backend databases.
• Frontend applications.

Docker Compose files are written in YAML. The most basic Docker Compose file for Deephaven with Python is:

services:
  deephaven:
    image: ghcr.io/deephaven/server:${VERSION:-latest}
    ports:
      - '${DEEPHAVEN_PORT:-10000}:10000'
    volumes:
      - ./data:/data
    environment:
      - START_OPTS=-Xmx4g

To make Deephaven accessible on a different port, set the DEEPHAVEN_PORT environment variable:

export DEEPHAVEN_PORT=10001

Or set it in the Docker Compose file:

services:
  deephaven:
    image: ghcr.io/deephaven/server:${VERSION:-latest}
    ports:
      - '${DEEPHAVEN_PORT:-10001}:10000'
    volumes:
      - ./data:/data
    environment:
      - START_OPTS=-Xmx4g

To make a Docker Compose application depend on an image built by a Dockerfile in the same directory, use the build key:

services:
  deephaven:
    build: .
    ports:
      - '${DEEPHAVEN_PORT:-10000}:10000'
    volumes:
      - ./data:/data
    environment:
      - START_OPTS=-Xmx4g

Running any Docker compose application is as simple as:

docker compose up

Deephaven offers a variety of pre-built Docker Compose files that can be used to quickly get up and running with Deephaven. They cover both the Python and Groovy API and add more services including example data, extended Python images, and Redpanda.

Related documentation

• Configure the production application
• Deephaven configuration files
• Build from source