Version: Python

Install and run with Docker

Run Deephaven from Docker

Deephaven can be run from pre-built Docker images and requires only Docker. This guide will teach you how to run Deephaven from Docker, choose a deployment, and customize it for your applications.

note

Docker isn't the only way to run Deephaven. Users who wish to use Python without Docker should install Deephaven with pip. Developers interested in tinkering with and modifying source code should build Deephaven from source. Users who wish to run from build artifacts without Docker should install and run the Deephaven production application.

Prerequisites

Building and running Deephaven requires a few software packages.

Package	Version	OS
docker	^20.10.8	All
Windows	10 (OS build 20262 or higher)	Only Windows
WSL	2	Only Linux via Windows

You can check if these packages are installed and functioning by running:

docker version
docker compose version
docker run hello-world

Installing WSL...

Deephaven can be run natively on Windows, without installing WSL. However, users who want to run Deephaven inside a GNU/Linux environment on a Windows machine will need Windows Subsystem for Linux (WSL) version 2. WSL is not needed on other operating systems.

Instructions for installing WSL 2 can be found at https://docs.microsoft.com/en-us/windows/wsl/install-win10. The latest Ubuntu Linux distribution for WSL 2 is recommended.

Installing Docker

Instructions for installing and configuring Docker can be found at https://docs.docker.com/get-docker/. Windows users should follow the WSL2 instructions.

Instructions for installing and configuring Docker Compose can be found here.

Docker RAM settings

Tests run as part of the build process require at least 4GB of Docker RAM. To check your Docker configuration, run:

docker info | grep Memory

By default, Docker on Mac is configured with 2 GB of RAM. If you need to increase the memory on your Mac, click on the Docker icon on the top bar and navigate to Preferences->Resources->Memory. Docker on Windows and Linux should not require configuration changes.

Docker WSL settings

If you are using WSL, Docker's settings must be configured to allow WSL access. In Docker Desktop, navigate to Settings->Resources->WSL Integration, and enable your distribution. After restarting your WSL shell, you can run Docker commands from WSL.

If docker run hello-world does not work...

If docker run hello-world does not work, try the following:

Is Docker running?
```
docker info
```

(Linux) Are you in the docker user group?

sudo groupadd docker
sudo usermod -aG docker $USER

We recommend using Docker Compose for customized deployments.

The simplest possible installation

The following shell command downloads and runs the server image:

docker run --name deephaven -p 10000:10000 ghcr.io/deephaven/server:latest

Replace server with any other deployment to change the pre-built image that gets downloaded and run.

danger

docker run creates a new container from an image every time it's called. To reuse a container created from docker run, use docker start.

note

This default configuration uses a pre-shared key to authenticate users. If not explicitly set, a randomly generated key gets printed to the Docker logs. See set a pre-shared key for how to set your own key.

Choose a deployment

Deephaven offers several pre-built Docker images that can be downloaded and used to run Deephaven:

server: Python
server-slim: Groovy
server-nltk: Python with NLTK
server-pytorch: Python with PyTorch
server-sklearn: Python with SciKit-Learn
server-tensorflow: Python with TensorFlow

Deephaven also has many pre-built docker-compose.yml files that build the images above with some additional features, including example data and Redpanda. They are located here.

note

The Python packages in the images above can be installed manually like any other Python package. For more information, see installing Python packages in Deephaven.

Image versions

The latest version is used in the examples below. This corresponds to the most recent release number (e.g. 0.33.3, 0.33.0, etc.). While it's recommended to stay up-to-date with recent releases, Deephaven has many releases that can be used if desired. Versions can be any of the following:

latest (default): The most recent release.
A specific release tag: 0.32.0, 0.33.3, etc.
edge: An image published nightly and contains unreleased features.

Modify the deployment

The Deephaven deployment can be modified through Docker alone or with Docker Compose. The subsections below present ways to modify the deployment using both. Deephaven recommends the using Docker Compose when creating custom deployments. See Key benefits of Docker Compose for more information on why.

Modifying the deployment with Docker should be done with docker create. docker run will always create a new container from an image. To run a pre-existing container, use docker start.

Modifying the deployment with Docker Compose requires updating the docker-compose.yml file used to build the container. The examples below will modify the following docker-compose.yaml file.

docker-compose.yml

services:
  deephaven:
    image: ghcr.io/deephaven/server:${VERSION:-latest}
    ports:
      - '${DEEPHAVEN_PORT:-10000}:10000'
    volumes:
      - ./data:/data
    environment:
      - START_OPTS=-Xmx4g

Set a pre-shared key

Deephaven, by default uses a pre-shared key to authenticate users looking to access Deephaven. If the key isn't set, Deephaven uses a randomly generated key that gets printed to the Docker logs.

The following deployment set the pre-shared key to YOUR_PASSWORD_HERE.

Sh
Yaml

docker create --name deephaven -p 10000:10000 --env START_OPTS=-Dauthentication.psk=YOUR_PASSWORD_HERE ghcr.io/deephaven/server:latest

services:
  deephaven:
    image: ghcr.io/deephaven/server:${VERSION:-latest}
    ports:
      - '${DEEPHAVEN_PORT:-10000}:10000'
    volumes:
      - ./data:/data
    environment:
      - START_OPTS=-Xmx4g -Dauthentication.psk=YOUR_PASSWORD_HERE

Disable authentication

Anonymous authentication allows anyone to access a Deephaven instance. The following deployment enable anonymous authentication.

Sh
Yaml

docker create --name deephaven -p 10000:10000 --env START_OPTS=-DAuthHandlers=io.deephaven.auth.AnonymousAuthenticationHandler ghcr.io/deephaven/server:latest

services:
  deephaven:
    image: ghcr.io/deephaven/server:${VERSION:-latest}
    ports:
      - '${DEEPHAVEN_PORT:-10000}:10000'
    volumes:
      - ./data:/data
    environment:
      - START_OPTS=-Xmx4g -DAuthHandlers=io.deephaven.auth.AnonymousAuthenticationHandler

Add more memory

The following deployment tell the server to allocate 8GB of heap memory instead of the default of 4GB.

Sh
Yaml

docker create --name deephaven -p 10000:10000 --env START_OPTS=-Xmx8g ghcr.io/deephaven/server:latest

services:
  deephaven:
    image: ghcr.io/deephaven/server:${VERSION:-latest}
    ports:
      - '${DEEPHAVEN_PORT:-10000}:10000'
    volumes:
      - ./data:/data
    environment:
      - START_OPTS=-Xmx8g

Change the port

The following deployment tell Deephaven to expose port 9999 for the user to connect to via their web browser.

Sh
Yaml

docker create --name deephaven -p 9999:10000 ghcr.io/deephaven/server:latest

services:
  deephaven:
    image: ghcr.io/deephaven/server:${VERSION:-latest}
    ports:
      - '${DEEPHAVEN_PORT:-9999}:10000'
    volumes:
      - ./data:/data
    environment:
      - START_OPTS=-Xmx4g

Add a second volume

Deephaven, by default, comes with a single data volume. The following deployment mounts a second specialty volume:

Sh
Yaml

docker create --name deephaven -p 10000:10000 -v ./specialty:/specialty ghcr.io/deephaven/server:latest

services:
  deephaven:
    image: ghcr.io/deephaven/server:${VERSION:-latest}
    ports:
      - '${DEEPHAVEN_PORT:-10000}:10000'
    volumes:
      - ./data:/data
      - ./specialty:/specialty
    environment:
      - START_OPTS=-Xmx4g

Add a second image

Docker Compose specializes in running multi-container applications. In fact, Deephaven used to run in four separate containers before being reduced to one. Adding a second image to a Docker application should always be done with Docker Compose. The following YAML file runs Deephaven with Redpanda.

services:
  deephaven:
    image: ghcr.io/deephaven/server:${VERSION:-latest}
    ports:
      - '${DEEPHAVEN_PORT:-10000}:10000'
    volumes:
      - ./data:/data
    environment:
      - START_OPTS=-Xmx4g

  redpanda:
    command:
      - redpanda
      - start
      - --kafka-addr internal://0.0.0.0:9092,external://0.0.0.0:19092
      - --advertise-kafka-addr internal://redpanda:9092,external://localhost:19092
      - --schema-registry-addr internal://0.0.0.0:8081,external://0.0.0.0:18081
      - --smp 1
      - --memory 1G
      - --mode dev-container
    image: docker.redpanda.com/redpandadata/redpanda:v23.2.18
    ports:
      - 8081:8081
      - 18081:18081
      - 9092:9092
      - 19092:19092

Build a custom image

Custom Docker deployments often require things that cannot be done in a YAML file or Docker command. For instance, it is not possible to install a Python package this way. Such deployments typically extend a Docker image using both a Dockerfile and a docker-compose.yml file.

The following subsections build a custom Deephaven application through Docker with several Python packages installed that do not ship with official Deephaven Docker images.

note

This example uses a flat directory structure - all files are placed in the same directory.

Dockerfile

A Dockerfile will dictate which Docker images to build containers from and what else distinguishes these containers from their standard counterparts.

The following Dockerfile takes the latest Deephaven server image and installs the Python packages defined in requirements.txt into the container created from it.

FROM ghcr.io/deephaven/server:latest
COPY requirements.txt /requirements.txt
RUN pip install -r /requirements.txt && rm /requirements.txt

docker-compose.yml

For this example, Docker Compose must be told to build a Docker image locally rather than pull one from the web. The following YAML file builds a Docker container from the instructions specified in the Dockerfile just created.

services:
  deephaven:
    build: .
    ports:
      - '${DEEPHAVEN_PORT:-10000}:10000'
    volumes:
      - ./data:/data
    environment:
      - START_OPTS=-Xmx4g

Start the application

To start a Deephaven application built this way, run:

docker compose up --build

The --build flag tells Docker to build the services specified by the docker-compose.yml file. The Dockerfile defines the custom installation process of the service.

Run Deephaven IDE

Once Deephaven is running, you can launch a Deephaven IDE in your web browser. The Deephaven IDE allows you to interactively analyze data and develop new analytics.

If Deephaven is running locally, navigate to http://localhost:10000/ide/.
If Deephaven is running remotely, navigate to http://<hostname>:10000/ide/, where <hostname> is the address of the machine Deephaven is running on.

Manage example data

The Deephaven examples repository contains data sets to help learn how to use Deephaven. Deephaven's documentation uses these data sets extensively, and they are needed to run some examples.

If you have chosen a deployment with example data, the example data sets will be downloaded to data/examples within your Deephaven folder, which translates to /data/examples within the Deephaven Docker container. See Docker data volumes for more information on how files get mounted in Docker.

What next?

With a Deephaven deployment running from Docker, check out any of these user guides to get started writing queries:

Quickstart

Run Deephaven from Docker​

Prerequisites​

The simplest possible installation​

Choose a deployment​

Image versions​

Modify the deployment​

Set a pre-shared key​

Disable authentication​

Add more memory​

Change the port​

Add a second volume​

Add a second image​

Build a custom image​

Dockerfile​

docker-compose.yml​

Start the application​

Run Deephaven IDE​

Manage example data​

What next?​

Related documentation​

Run Deephaven from Docker

Prerequisites

The simplest possible installation

Choose a deployment

Image versions

Modify the deployment

Set a pre-shared key

Disable authentication

Add more memory

Change the port

Add a second volume

Add a second image

Build a custom image

Dockerfile

docker-compose.yml

Start the application

Run Deephaven IDE

Manage example data

What next?

Related documentation