Skip to main content
Version: Python

Build and run Deephaven from source code

This guide will show you how to build and launch Deephaven Community Core from source code. It provides a starting point for tinkerers and developers who want to dig into configuration or experiment with code changes. These instructions show how to build on multiple operating systems, including Linux, Windows, and Mac.

tip

Launching from source code is recommended for users who wish to tinker with and modify source code. For an easier installation method, see Launch Deephaven from pre-built images.

Prerequisites​

Required dependencies​

Building and running Deephaven requires a few software packages.

PackageVersionOS
git^2.25.0All
java>=11, <20All
docker^20.10.8All
Windows10 (OS build 20262 or higher)Only Windows
WSL2Only Linux via Windows

To check if these packages are installed and functioning, run:

git version
java -version
docker version
docker compose version
docker run hello-world
note

Internally, the Java build process will use Gradle Auto Provisioning to download and use the appropriate Java version for building and testing.

Installing WSL...

Deephaven can be run natively on Windows without installing WSL. However, users who want to run Deephaven inside a GNU/Linux environment on a Windows machine will need Windows Subsystem for Linux (WSL) version 2. WSL is not needed on other operating systems.

Instructions for installing WSL 2 can be found at https://learn.microsoft.com/en-us/windows/wsl/install. The latest Ubuntu Linux distribution for WSL 2 is recommended.

Installing Java

Deephaven can be built with Oracle OpenJDK, Adoptium OpenJDK, and Zulu OpenJDK. Other flavors of the OpenJDK will likely work as well.

You can choose to install with the vendor links above, or you may prefer to use your package manager's version, which is often the easiest route:

  • Debian / Ubuntu / Windows WSL2 Ubuntu

    apt-get update
    apt-get install openjdk-11-jdk

    See debian or Ubuntu for more information.

  • Fedora / RedHat

    dnf install java-11-openjdk-devel

    See Fedora or RedHat for more information.

  • Mac

    brew install openjdk@11

OpenJDK 11 may need to be added to your path. For Intel Macs:

echo 'export PATH="/usr/local/opt/openjdk@11/bin:$PATH"' >> ~/.zshrc

For M1 Macs:

  echo 'export PATH="/opt/homebrew/opt/openjdk@11/bin:$PATH"' >> ~/.zshrc

See Brew for more information.

Installing Docker

Instructions for installing and configuring Docker can be found at https://docs.docker.com/get-docker/. Windows users should follow the WSL2 instructions.

Instructions for installing and configuring docker compose can be found at https://docs.docker.com/compose/install/.

Docker RAM settings

Tests run as part of the build process require at least 4GB of Docker RAM. To check your Docker configuration, run:

docker info | grep Memory

By default, Docker on Mac is configured with 2 GB of RAM. To increase the memory on your Mac, click on the Docker icon on the top bar and navigate to Preferences->Resources->Memory. Docker on Windows and Linux should not require configuration changes.

img

Docker WSL settings

If using WSL, you need to change Docker's configuration settings to allow WSL access. In Docker Desktop, navigate to Settings->Resources->WSL Integration, and enable your distribution. After restarting your WSL shell, you will be able to run Docker commands from WSL.

img

If docker run hello-world does not work...

If docker run hello-world does not work, try the following:

  1. Is Docker running?

    docker info

  2. (Linux) Are you in the docker user group?

    sudo groupadd docker
    sudo usermod -aG docker $USER

For the new group to apply, you must finish your current session and start a new one (log out and log in). Also, if the Gradle daemon is already running for your user, it needs to be stopped since it is running with the old credentials.

If `git clone` fails on WSL 2...

WSL 2 has a known bug that results in git clone failures in some environments. The bug has been reported since 2019. You may be able to fix this as follows.

Update networking drivers:

  1. On Windows, open the Device Manager.
  2. Expand "Network adapters".
  3. Find which network device you are using (WiFi or wired), and note the brand.
  4. Google "newest Windows 10 device drivers for <brand_name>".
  5. Install the drivers.
  6. Restart.

Now you need to set the maximum network packet size, known as the maximum transmission unit (MTU), to something slightly smaller than the WSL interface value.

In powershell, look up the current MTU for the WSL interface:

netsh interface ipv4 show subinterface

You will see output that looks like this:

   MTU  MediaSenseState   Bytes In  Bytes Out  Interface
------  ---------------  ---------  ---------  -------------
4294967295                1          0       5150  Loopback Pseudo-Interface 1
  1500                1   83773143   12179977  Wi-Fi
  1500                5          0          0  Ethernet
  1500                5          0          0  Local Area Connection* 1
  1500                5          0          0  Local Area Connection* 2
  1500                5          0          0  Ethernet 2
  1500                1      29246   11502767  vEthernet (WSL)

Note the vEthernet interface's MTU.

In a WSL 2 Ubuntu shell, set the MTU to a number slightly smaller than the WSL vEthernet value obtained above. This ensures that there is enough buffer to wrap the packets:

sudo ip link set dev eth0 mtu 1350

Build and run Deephaven​

The following instructions are a condensed version of instructions found in the deephaven-core repository. For the full instructions with explanations of configuration parameters, SSL, and more, see the README.

note

The following steps show how to build and run Deephaven with Python from source. For Groovy, see here, or click the Language drop-down menu at the top left of the webpage and select Java (Groovy).

Clone the deephaven-core repository​

Once all of the required dependencies are installed and functioning, run:

git clone https://github.com/deephaven/deephaven-core.git
cd deephaven-core

The first of these lines clones the repository; the second line sets the directory to the new local clone.

Set up the Python virtual environment​

First, set up a virtual environment.

python -m venv /tmp/my-dh-venv
source /tmp/my-dh-venv/bin/activate

Build and install the wheel​

Then, build and install the wheel.

./gradlew py-server:assemble

pip install --find-links py/server/build/wheel "deephaven-core[autocomplete]"

Build and run​

Lastly, build and run Deephaven.

./gradlew server-jetty-app:run

Run Deephaven IDE​

Once Deephaven is running, you can launch a Deephaven IDE in your web browser. Deephaven IDE allows you to interactively analyze data and develop new analytics.

  • If Deephaven is running locally, navigate to http://localhost:10000/ide/.
  • If Deephaven is running remotely, navigate to http://<hostname>:10000/ide/, where <hostname> is the address of the machine Deephaven is running on.

Authentication​

Deephaven, by default, uses pre-shared key authentication. If no key is set, a randomly generated key will be used to log into the server each time it starts. The randomly generated key is printed to the Docker logs like this:

img

To set your own pre-shared key, add -Ppsk=<YourPasswordHere>:

./gradlew server-jetty-app:run -Ppsk=YOUR_PASSWORD_HERE

The pre-shared key is printed to the Docker log like this:

img