Skip to main content
Version: Python

Build and launch from source

warning

This page is a guide for building and running Deephaven from source code. These instructions are for users and developers interested in tinkering with and modifying Deephaven source code.

Almost all users will want to Launch Deephaven from pre-built images. It is the easiest way to deploy.

If you are not sure which of the two is right for you, use the pre-built images. For detailed instructions on how to do this, see Launch Deephaven from pre-built images.

Deephaven Community Core is a real-time, time-series, column-oriented analytics engine with relational database features. Queries can seamlessly operate upon both historical and real-time data. It can ingest data from a variety of sources, apply computation and analysis algorithms to that data, and build rich queries, dashboards, and representations with the results.

In this tutorial, you'll learn how to set up and launch Deephaven.

Prerequisites​

Required dependencies​

Building and running Deephaven requires a few software packages.

PackageVersionOS
git^2.25.0All
java>=11All
docker^20.10.8All
docker-compose^1.29.0All
Windows10 (OS build 20262 or higher)Only Windows
WSL2Only Windows

You can check if these packages are installed and functioning by running:

git version
java -version
docker version
docker-compose version
docker run hello-world
note

Internally, the Java build process will use Gradle Auto Provisioning to download and use the appropriate Java version for building and testing.

On Windows, all commands must be run inside a WSL 2 terminal.

Installing WSL...

On Windows, Windows Subsystem for Linux (WSL) version 2 must be installed. WSL is not needed on other operating systems.

Instructions for installing WSL 2 can be found at https://docs.microsoft.com/en-us/windows/wsl/install-win10. The latest Ubuntu Linux distribution for WSL 2 is recommended.

Installing Java

Deephaven can be built with Oracle OpenJDK, Adoptium OpenJDK, Zulu OpenJDK, and likely other flavors of the OpenJDK.

You can choose to install with the vendor links above, or you may prefer to use your package manager's version, which is often the easiest route:

  • Debian / Ubuntu / Windows WSL2 Ubuntu

    apt-get update
    apt-get install openjdk-11-jdk-headless

    See debian or Ubuntu for more information.

  • Fedora / RedHat

    dnf install java-11-openjdk-devel

    See Fedora or RedHat for more information.

  • Mac

    brew install [email protected]

OpenJDK 11 may need to be added to your path. For Intel Macs:

echo 'export PATH="/usr/local/opt/[email protected]/bin:$PATH"' >> ~/.zshrc

For M1 Macs:

  echo 'export PATH="/opt/homebrew/opt/[email protected]/bin:$PATH"' >> ~/.zshrc

See Brew for more information.

Installing Docker

Instructions for installing and configuring Docker can be found at https://docs.docker.com/get-docker/. Windows users should follow the WSL2 instructions.

Instructions for installing and configuring docker-compose can be found at https://docs.docker.com/compose/install/.

Docker RAM settings

Tests run as part of the build process require at least 4GB of Docker RAM. To check your Docker configuration, run:

docker info | grep Memory

By default, Docker on Mac is configured with 2 GB of RAM. If you need to increase the memory on your Mac, click on the Docker icon on the top bar and navigate to Preferences->Resources->Memory. Docker on Windows and Linux should not require configuration changes.

img

Docker WSL settings

On Windows, Docker must be configured to allow WSL to access Docker. In Docker Desktop, navigate to Settings->Resources->WSL Integration, and enable your distribution. After restarting your WSL shell, you will be able to run Docker commands from WSL.

img

If docker run hello-world does not work...

If docker run hello-world does not work, try the following:

  1. Is Docker running?

    docker info

  2. (Linux) Are you in the docker user group?

    sudo groupadd docker
    sudo usermod -aG docker $USER

In order for the new group to apply, you must finish your current session and start a new one (log out and log in). Also, if the Gradle daemon is already running for your user, it needs to be stopped since it is running with the old credentials.

If `git clone` fails on WSL 2...

WSL 2 has a known bug that results in git clone failures in some environments. The bug has been reported since 2019. You may be able to fix this as follows.

Update networking drivers:

  1. On Windows, open the Device Manager.
  2. Expand "Network adapters".
  3. Find which network device you are using (wifi, or wired), and note the brand.
  4. Google "newest Windows 10 device drivers for <brand_name>".
  5. Install the drivers.
  6. Restart.

Now you need to set the maximum network packet size, known as the maximum transmission unit (MTU), to something slightly smaller than the WSL interface value.

In powershell, lookup the current MTU for the WSL interface:

netsh interface ipv4 show subinterface

You will see output that looks like:

   MTU  MediaSenseState   Bytes In  Bytes Out  Interface
------  ---------------  ---------  ---------  -------------
4294967295                1          0       5150  Loopback Pseudo-Interface 1
  1500                1   83773143   12179977  Wi-Fi
  1500                5          0          0  Ethernet
  1500                5          0          0  Local Area Connection* 1
  1500                5          0          0  Local Area Connection* 2
  1500                5          0          0  Ethernet 2
  1500                1      29246   11502767  vEthernet (WSL)

Note the vEthernet interface's MTU.

In a WSL 2 Ubuntu shell, set the MTU to a number slightly smaller than the WSL vEthernet value obtained above. This ensures that there is enough buffer to wrap the packets:

sudo ip link set dev eth0 mtu 1350

Checkout and build Deephaven​

Once all of the required dependencies are installed and functioning, run:

git clone https://github.com/deephaven/deephaven-core.git
cd deephaven-core
./gradlew prepareCompose

These commands will create:

  1. A deephaven-core directory containing the source code.
  2. Docker images containing everything needed to launch Deephaven.

Run Deephaven​

note

Commands in the following sections for interacting with a deployment must be run from the project's root directory, deephaven-core.

important

On an M1 Mac, Deephaven may fail to start if one of the public images was used previously. Stop and remove all Docker containers, then run docker system prune -a to remove the image.

Python​

From the deephaven-core directory, run

docker-compose up

This will start Deephaven. The console will fill with status and logging output.

Killing the process (e.g. Ctrl+C) will stop Deephaven.

Groovy​

From the deephaven-core directory, run

docker-compose --env-file default_groovy.env up

This will start Deephaven. The console will fill with status and logging output.

Killing the process (e.g., Ctrl+C) will stop Deephaven.

Run Deephaven IDE​

Once Deephaven is running, you can launch a Deephaven IDE in your web browser. Deephaven IDE allows you to interactively analyze data and develop new analytics.

  • If Deephaven is running locally, navigate to http://localhost:10000/ide/.
  • If Deephaven is running remotely, navigate to http://<hostname>:10000/ide/, where <hostname> is the address of the machine Deephaven is running on.