Install and run with Docker
Run Deephaven from Docker
Deephaven can be run from pre-built Docker images and requires only Docker. This guide will teach you how to run Deephaven from Docker, choose a deployment, and customize it for your applications.
Docker isn't the only way to run Deephaven. Users who wish to use Python without Docker should install Deephaven with pip. Developers interested in tinkering with and modifying source code should build Deephaven from source. Users who wish to run from build artifacts without Docker should install and run Deephaven natively.
Prerequisites
Building and running Deephaven requires a few software packages.
Package | Version | OS |
---|---|---|
docker | ^20.10.8 | All |
Windows | 10 (OS build 20262 or higher) | Only Windows |
WSL | 2 | Only Linux via Windows |
You can check if these packages are installed and functioning by running:
docker version
docker compose version
docker run hello-world
Installing WSL...
Deephaven can be run natively on Windows, without installing WSL. However, users who want to run Deephaven inside a GNU/Linux environment on a Windows machine will need Windows Subsystem for Linux (WSL) version 2. WSL is not needed on other operating systems.
Instructions for installing WSL 2 can be found at https://docs.microsoft.com/en-us/windows/wsl/install-win10. The latest Ubuntu Linux distribution for WSL 2 is recommended.
Installing Docker
Instructions for installing and configuring Docker can be found at https://docs.docker.com/get-docker/. Windows users should follow the WSL2 instructions.
Instructions for installing and configuring Docker Compose can be found here.
Docker RAM settings
Tests run as part of the build process require at least 4GB of Docker RAM. To check your Docker configuration, run:
docker info | grep Memory
By default, Docker on Mac is configured with 2 GB of RAM. If you need to increase the memory on your Mac, click on the Docker icon on the top bar and navigate to Preferences->Resources->Memory
. Docker on Windows and Linux should not require configuration changes.
Docker WSL settings
If you are using WSL, Docker's settings must be configured to allow WSL access. In Docker Desktop, navigate to Settings->Resources->WSL Integration
, and enable your distribution. After restarting your WSL shell, you can run Docker commands from WSL.
If docker run hello-world
does not work...
If docker run hello-world
does not work, try the following:
docker info
(Linux) Are you in the
docker
user group?sudo groupadd docker
sudo usermod -aG docker $USER
We recommend using Docker Compose for customized deployments.
The simplest possible installation
The following shell command downloads and runs the server
image:
docker run --name deephaven -p 10000:10000 ghcr.io/deephaven/server:latest
Replace server
with any other deployment to change the pre-built image that gets downloaded and run.
docker run
creates a new container from an image every time it's called. To reuse a container created from docker run
, use docker start
.
This default configuration uses a pre-shared key to authenticate users. If not explicitly set, a randomly generated key gets printed to the Docker logs. See set a pre-shared key for how to set your own key.
Choose a deployment
Deephaven offers several pre-built Docker images that can be downloaded and used to run Deephaven:
server
: Pythonserver-slim
: Groovyserver-nltk
: Python with NLTKserver-pytorch
: Python with PyTorchserver-sklearn
: Python with SciKit-Learnserver-tensorflow
: Python with TensorFlow
Deephaven also has many pre-built docker-compose.yml
files that build the images above with some additional features, including example data and Redpanda. They are located here.
The Python packages in the images above can be installed manually like any other Python package. For more information, see installing Python packages in Deephaven.
Image versions
The latest
version is used in the examples below. This corresponds to the most recent release number (e.g. 0.33.3
, 0.33.0
, etc.). While it's recommended to stay up-to-date with recent releases, Deephaven has many releases that can be used if desired. Versions can be any of the following:
latest
(default): The most recent release.- A specific release tag:
0.32.0
,0.33.3
, etc. edge
: An image published nightly and contains unreleased features.
Modify the deployment
The Deephaven deployment can be modified through Docker alone or with Docker Compose. The subsections below present ways to modify the deployment using both. Deephaven recommends the using Docker Compose when creating custom deployments. See Key benefits of Docker Compose for more information on why.
Modifying the deployment with Docker should be done with docker create
. docker run
will always create a new container from an image. To run a pre-existing container, use docker start
.
Modifying the deployment with Docker Compose requires updating the docker-compose.yml
file used to build the container. The examples below will modify the following docker-compose.yaml
file.
docker-compose.yml
services:
deephaven:
image: ghcr.io/deephaven/server:${VERSION:-latest}
ports:
- '${DEEPHAVEN_PORT:-10000}:10000'
volumes:
- ./data:/data
environment:
- START_OPTS=-Xmx4g
Set a pre-shared key
Deephaven, by default uses a pre-shared key to authenticate users looking to access Deephaven. If the key isn't set, Deephaven uses a randomly generated key that gets printed to the Docker logs.
The following deployment set the pre-shared key to YOUR_PASSWORD_HERE
.
- Sh
- Yaml
docker create --name deephaven -p 10000:10000 --env START_OPTS=-Dauthentication.psk=YOUR_PASSWORD_HERE ghcr.io/deephaven/server:latest
services:
deephaven:
image: ghcr.io/deephaven/server:${VERSION:-latest}
ports:
- '${DEEPHAVEN_PORT:-10000}:10000'
volumes:
- ./data:/data
environment:
- START_OPTS=-Xmx4g -Dauthentication.psk=YOUR_PASSWORD_HERE
Disable authentication
Anonymous authentication allows anyone to access a Deephaven instance. The following deployment enable anonymous authentication.
- Sh
- Yaml
docker create --name deephaven -p 10000:10000 --env START_OPTS=-DAuthHandlers=io.deephaven.auth.AnonymousAuthenticationHandler ghcr.io/deephaven/server:latest
services:
deephaven:
image: ghcr.io/deephaven/server:${VERSION:-latest}
ports:
- '${DEEPHAVEN_PORT:-10000}:10000'
volumes:
- ./data:/data
environment:
- START_OPTS=-Xmx4g -DAuthHandlers=io.deephaven.auth.AnonymousAuthenticationHandler
Add more memory
The following deployment tell the server to allocate 8GB of heap memory instead of the default of 4GB.
- Sh
- Yaml
docker create --name deephaven -p 10000:10000 --env START_OPTS=-Xmx8g ghcr.io/deephaven/server:latest
services:
deephaven:
image: ghcr.io/deephaven/server:${VERSION:-latest}
ports:
- '${DEEPHAVEN_PORT:-10000}:10000'
volumes:
- ./data:/data
environment:
- START_OPTS=-Xmx8g
Change the port
The following deployment tell Deephaven to expose port 9999
for the user to connect to via their web browser.
- Sh
- Yaml
docker create --name deephaven -p 9999:10000 ghcr.io/deephaven/server:latest
services:
deephaven:
image: ghcr.io/deephaven/server:${VERSION:-latest}
ports:
- '${DEEPHAVEN_PORT:-9999}:10000'
volumes:
- ./data:/data
environment:
- START_OPTS=-Xmx4g
Add a second volume
Deephaven, by default, comes with a single data
volume. The following deployment mounts a second specialty
volume:
- Sh
- Yaml
docker create --name deephaven -p 10000:10000 -v ./specialty:/specialty ghcr.io/deephaven/server:latest
services:
deephaven:
image: ghcr.io/deephaven/server:${VERSION:-latest}
ports:
- '${DEEPHAVEN_PORT:-10000}:10000'
volumes:
- ./data:/data
- ./specialty:/specialty
environment:
- START_OPTS=-Xmx4g
Add a second image
Docker Compose specializes in running multi-container applications. In fact, Deephaven used to run in four separate containers before being reduced to one. Adding a second image to a Docker application should always be done with Docker Compose. The following YAML file runs Deephaven with Redpanda.
services:
deephaven:
image: ghcr.io/deephaven/server:${VERSION:-latest}
ports:
- '${DEEPHAVEN_PORT:-10000}:10000'
volumes:
- ./data:/data
environment:
- START_OPTS=-Xmx4g
redpanda:
command:
- redpanda
- start
- --kafka-addr internal://0.0.0.0:9092,external://0.0.0.0:19092
- --advertise-kafka-addr internal://redpanda:9092,external://localhost:19092
- --schema-registry-addr internal://0.0.0.0:8081,external://0.0.0.0:18081
- --smp 1
- --memory 1G
- --mode dev-container
image: docker.redpanda.com/redpandadata/redpanda:v23.2.18
ports:
- 8081:8081
- 18081:18081
- 9092:9092
- 19092:19092
Build a custom image
Custom Docker deployments often require things that cannot be done in a YAML file or Docker command. For instance, it is not possible to install a Python package this way. Such deployments typically extend a Docker image using both a Dockerfile and a docker-compose.yml file.
The following subsections build a custom Deephaven application through Docker with several Python packages installed that do not ship with official Deephaven Docker images.
This example uses a flat directory structure - all files are placed in the same directory.
Dockerfile
A Dockerfile will dictate which Docker images to build containers from and what else distinguishes these containers from their standard counterparts.
The following Dockerfile takes the latest Deephaven server
image and installs the Python packages defined in requirements.txt
into the container created from it.
FROM ghcr.io/deephaven/server:latest
COPY requirements.txt /requirements.txt
RUN pip install -r /requirements.txt && rm /requirements.txt
docker-compose.yml
For this example, Docker Compose must be told to build a Docker image locally rather than pull one from the web. The following YAML file builds a Docker container from the instructions specified in the Dockerfile just created.
services:
deephaven:
build: .
ports:
- '${DEEPHAVEN_PORT:-10000}:10000'
volumes:
- ./data:/data
environment:
- START_OPTS=-Xmx4g
Start the application
To start a Deephaven application built this way, run:
docker compose up --build
The --build
flag tells Docker to build the services specified by the docker-compose.yml
file. The Dockerfile
defines the custom installation process of the service.
Run Deephaven IDE
Once Deephaven is running, you can launch a Deephaven IDE in your web browser. The Deephaven IDE allows you to interactively analyze data and develop new analytics.
- If Deephaven is running locally, navigate to http://localhost:10000/ide/.
- If Deephaven is running remotely, navigate to
http://<hostname>:10000/ide/
, where<hostname>
is the address of the machine Deephaven is running on.
Manage example data
The Deephaven examples repository contains data sets to help learn how to use Deephaven. Deephaven's documentation uses these data sets extensively, and they are needed to run some examples.
If you have chosen a deployment with example data, the example data sets will be downloaded to data/examples
within your Deephaven folder, which translates to /data/examples
within the Deephaven Docker container. See Docker data volumes for more information on how files get mounted in Docker.
What next?
With a Deephaven deployment running from Docker, check out any of these user guides to get started writing queries: