Quick start
Install Deephaven from pre-built images
Deephaven can be downloaded in pre-built Docker images and requires only Docker to run. In this tutorial, you'll learn how to choose a Docker Compose configuration and use it to run Deephaven.
note
Developers interested in tinkering with and modifying Deephaven source code should follow the instructions in the build from source guide. Python users can also choose to install Deephaven using pip, and can follow along with the Quick start for pip.
TL;DR
Get started with the Python base image using these three CLI commands:
curl https://raw.githubusercontent.com/deephaven/deephaven-core/main/containers/python-examples/base/docker-compose.yml -O
docker-compose pull
docker-compose up
Now go to http://localhost:10000/ide/. You're up and running! For prerequisites, troubleshooting, alternate base images, and what to do next, read on.
important
On M1 Macs, you must build and launch from source code. A Docker-emulation bug leads to failures when running pre-built images.
Prerequisites
Building and running Deephaven requires a few software packages.
| Package | Version | OS |
|---|---|---|
| docker | ^20.10.8 | All |
| docker-compose | ^1.29.0 | All |
| Windows | 10 (OS build 20262 or higher) | Only Windows |
| WSL | 2 | Only Windows |
You can check if these packages are installed and functioning by running:
docker version
docker-compose version
docker run hello-world
note
On Windows, all commands must be run inside a WSL 2 terminal.
Installing WSL...
On Windows, Windows Subsystem for Linux (WSL) version 2 must be installed. WSL is not needed on other operating systems.
Instructions for installing WSL 2 can be found at https://docs.microsoft.com/en-us/windows/wsl/install-win10. The latest Ubuntu Linux distribution for WSL 2 is recommended.
Installing Docker
Instructions for installing and configuring Docker can be found at https://docs.docker.com/get-docker/. Windows users should follow the WSL2 instructions.
Instructions for installing and configuring docker-compose can be found at
https://docs.docker.com/compose/install/.
Docker RAM settings
Tests run as part of the build process require at least 4GB of Docker RAM. To check your Docker configuration, run:
docker info | grep Memory
By default, Docker on Mac is configured with 2 GB of RAM. If you need to increase the memory on your Mac, click on the Docker icon on the top bar and navigate to Preferences->Resources->Memory. Docker on Windows and Linux should not require configuration changes.
Docker WSL settings
On Windows, Docker must be configured to allow WSL to access Docker. In Docker Desktop, navigate to Settings->Resources->WSL Integration, and enable your distribution. After restarting your WSL shell, you will be able to run Docker commands from WSL.
If docker run hello-world does not work...
If docker run hello-world does not work, try the following:
docker info(Linux) Are you in the
dockeruser group?sudo groupadd docker
sudo usermod -aG docker $USER
Choose a deployment
When determining which deployment is right for your application, there are three key questions:
- What programming language will your queries be written in?
- Do you need example data from the Deephaven's examples repository?
- Do you plan to use one of our machine learning images?
Based on your answers, you can use the following table to find the URL to the desired Docker Compose configuration. For example, if you will be working through examples in the Deephaven documentation, and you develop in Python, you will choose https://raw.githubusercontent.com/deephaven/deephaven-core/main/containers/python-examples/base/docker-compose.yml, since it supports Python queries and has the example data used in the Deephaven documentation.
Choose a version
The following commands default to running the latest release of Deephaven. To select other Deephaven versions, set the VERSION environment variable before running docker-compose commands.
VERSION can be set to:
latestto get the latest release. (default)- A specific release tag (e.g.,
0.4.0or0.4.1). edgeto get the images from the latest commit to the main branch.
For example, in Bash, configure the edge release by running:
export VERSION=edge
Set up your Deephaven deployment
First, create a directory for the system to live in. Use any directory name you like; we chose deephaven-deployment:
mkdir deephaven-deployment
Then, make that the current working directory:
cd deephaven-deployment
note
Commands in the following sections for interacting with a deployment must be run from the deployment directory.
Now, use curl to get the Docker Compose file for your desired configuration. Substitute the URL of your choice from the table above. We use the Python build with the examples manager included:
# Choose your compose file selected above.
compose_file=https://raw.githubusercontent.com/deephaven/deephaven-core/main/containers/python-examples/base/docker-compose.yml
curl -O "${compose_file}"
Now that the docker-compose.yml file is locally available, download the Docker images:
docker-compose pull
Since this step only gets the container images and does not run anything, the Deephaven services will not start, and you will not see any logging output.
caution
When new features are added to Deephaven, you will need to redownload the docker-compose.yml file to get the latest version of Deephaven.
Manage the Deephaven deployment
Now that your chosen configuration is set up, enter its directory and bring up the deployment:
docker-compose up -d
The -d option causes the containers to run in the background, in detached mode. This option allows you to use your shell after Docker launches the containers.
Since the container is running detached, you will not see any logs. However, you can follow the logs by running:
docker-compose logs -f
To stop running your containers without removing them, you can run the following:
docker-compose stop
tip
To start over on a fresh session, bring down your Docker image; e.g., docker-compose down.
Use CTRL+C to stop monitoring the logs and return to a prompt.
The deployment can be brought down by running:
docker-compose down
The Deephaven containers use a few Docker volumes to store persistent data. If you don't want to keep that persistent storage around, you might want to remove all the volumes that were associated with the deployment. This can be done by running:
warning
Running the following command will permanently delete important state for your Deephaven deployment. Only perform this step if you are certain that the deployment state is no longer needed.
docker-compose down -v
Manage example data
The Deephaven examples repository contains data sets that are useful when learning to use Deephaven. These data sets are used extensively in Deephaven's documentation and are needed to run some examples.
If you have chosen a deployment with example data, the example data sets will be downloaded. Production deployments containing your own data will not need the example data sets.
Deephaven's examples repository contains documentation on the available data sets. Additionally, there is documentation on managing the data sets. This includes instructions on how to upgrade to the latest version. See our guide, Docker data volumes, for more information on where these files reside.
Run Deephaven IDE
Once Deephaven is running, you can launch a Deephaven IDE in your web browser. The Deephaven IDE allows you to interactively analyze data and develop new analytics.
- If Deephaven is running locally, navigate to http://localhost:10000/ide/.
- If Deephaven is running remotely, navigate to
http://<hostname>:10000/ide/, where<hostname>is the address of the machine Deephaven is running on.
Run your first query
From the Deephaven IDE, you can perform your first query.
This script creates two small tables: one for employees and one for departments. It joins the two tables on the DeptID column to show the name of the department where each employee works.
from deephaven import new_table
from deephaven.column import string_col, int_col
from deephaven.constants import NULL_INT
left = new_table([
string_col("LastName", ["Rafferty", "Jones", "Steiner", "Robins", "Smith", "Rogers"]),
int_col("DeptID", [31, 33, 33, 34, 34, NULL_INT]),
string_col("Telephone", ["(347) 555-0123", "(917) 555-0198", "(212) 555-0167", "(952) 555-0110", None, None])
])
right = new_table([
int_col("DeptID", [31, 33, 34, 35]),
string_col("DeptName", ["Sales", "Engineering", "Clerical", "Marketing"]),
string_col("Telephone", ["(646) 555-0134", "(646) 555-0178", "(646) 555-0159", "(212) 555-0111"])
])
table = left.join(table=right, on=["DeptID"], joins=["DeptName,DeptTelephone=Telephone"])
- left
- right
- table