Skip to main content

Build and launch from source

warning

This page is a guide for building and running Deephaven from source code. These instructions are for developers interested in tinkering with and modifying Deephaven source code.

Almost all users will want to Launch Deephaven from pre-built images. It is the easiest way to deploy.

If you are not sure which of the two is right for you, use the pre-built images. For detailed instructions on how to do this, see Launch Deephaven from pre-built images.

Deephaven Community Core is a real-time, time-series, column-oriented analytics engine with relational database features. Queries can seamlessly operate upon both historical and real-time data. It can ingest data from a variety of sources, apply computation and analysis algorithms to that data, and build rich queries, dashboards, and representations with the results.

In this tutorial, you'll learn how to set up and launch Deephaven.

Prerequisites#

Required dependencies#

Building and running Deephaven requires a few software packages.

PackageVersionOS
git^2.25.0All
java~1.8.0All
docker^20.10.8All
docker-compose^1.29.0All
Windows10 (OS build 20262 or higher)Only Windows
WSL2Only Windows

You can check if these packages are installed and functioning by running:

git versionjava -versiondocker versiondocker-compose versiondocker run hello-world
note

On Windows, all commands must be run inside a WSL 2 terminal.

Installing WSL...

On Windows, Windows Subsystem for Linux (WSL) version 2 must be installed. WSL is not needed on other operating systems.

Instructions for installing WSL 2 can be found at https://docs.microsoft.com/en-us/windows/wsl/install-win10. The latest Ubuntu Linux distribution for WSL 2 is recommended.

Installing Java

Deephaven can be built with either Oracle JDK or OpenJDK. Java 8 is required.

To install Java, run:

  • Mac

    brew install openjdk@8

    OpenJDK 8 may need to be added to your path:

    echo 'export PATH="/usr/local/opt/openjdk@8/bin:$PATH"' >> ~/.zshrc
  • Windows WSL2 - Ubuntu

    sudo apt updatesudo apt install openjdk-8-jdk-headless
  • Linux

    sudo apt updatesudo apt install openjdk-8-jdk

    or

    sudo yum install java-1.8.0-openjdk
Installing Docker

Instructions for installing and configuring Docker can be found at https://docs.docker.com/get-docker/. Windows users should follow the WSL2 instructions.

Instructions for installing and configuring docker-compose can be found at https://docs.docker.com/compose/install/.

Docker RAM settings

Tests run as part of the build process require at least 4GB of Docker RAM. To check your Docker configuration, run:

docker info | grep Memory

By default, Docker on Mac is configured with 2 GB of RAM. If you need to increase the memory on your Mac, click on the Docker icon on the top bar and navigate to Preferences->Resources->Memory. Docker on Windows and Linux should not require configuration changes.

img

Docker WSL settings

On Windows, Docker must be configured to allow WSL to access Docker. In Docker Desktop, navigate to Settings->Resources->WSL Integration, and enable your distribution. After restarting your WSL shell, you will be able to run Docker commands from WSL.

img

If docker run hello-world does not work...

If docker run hello-world does not work, try the following:

  1. Is Docker running?

    docker info
  2. (Linux) Are you in the docker user group?

    sudo groupadd dockersudo usermod -aG docker $USER
If `git clone` fails on WSL 2...

WSL 2 has a known bug that results in git clone failures in some environments. The bug has been reported since 2019. You may be able to fix this as follows.

Update networking drivers:

  1. On Windows, open the Device Manager.
  2. Expand "Network adapters".
  3. Find which network device you are using (wifi, or wired), and note the brand.
  4. Google "newest Windows 10 device drivers for <brand_name>".
  5. Install the drivers.
  6. Restart.

Now you need to set the maximum network packet size, known as the maximum transmission unit (MTU), to something slightly smaller than the WSL interface value.

In powershell, lookup the current MTU for the WSL interface:

netsh interface ipv4 show subinterface

You will see output that looks like:

   MTU  MediaSenseState   Bytes In  Bytes Out  Interface------  ---------------  ---------  ---------  -------------4294967295                1          0       5150  Loopback Pseudo-Interface 1  1500                1   83773143   12179977  Wi-Fi  1500                5          0          0  Ethernet  1500                5          0          0  Local Area Connection* 1  1500                5          0          0  Local Area Connection* 2  1500                5          0          0  Ethernet 2  1500                1      29246   11502767  vEthernet (WSL)

Note the vEthernet interface's MTU.

In a WSL 2 Ubuntu shell, set the MTU to a number slightly smaller than the WSL vEthernet value obtained above. This ensures that there is enough buffer to wrap the packets:

sudo ip link set dev eth0 mtu 1350  

Checkout and build Deephaven#

Once all of the required dependencies are installed and functioning, run:

git clone https://github.com/deephaven/deephaven-core.gitcd deephaven-core./gradlew prepareCompose

These commands will create:

  1. A deephaven-core directory containing the source code.
  2. Docker images containing everything needed to launch Deephaven.

Run Deephaven#

note

Commands in the following sections for interacting with a deployment must be run from the project's root directory, deephaven-core.

Python#

From the deephaven-core directory, run

docker-compose up

This will start Deephaven. The console will fill with status and logging output.

Killing the process (e.g. Ctrl+C) will stop Deephaven.

Groovy#

From the deephaven-core directory, run

docker-compose --env-file default_groovy.env up

This will start Deephaven. The console will fill with status and logging output.

Killing the process (e.g., Ctrl+C) will stop Deephaven.

Run Deephaven IDE#

Once Deephaven is running, you can launch a Deephaven IDE in your web browser. Deephaven IDE allows you to interactively analyze data and develop new analytics.

  • If Deephaven is running locally, navigate to http://localhost:10000/ide/.
  • If Deephaven is running remotely, navigate to http://<hostname>:10000/ide/, where <hostname> is the address of the machine Deephaven is running on.

Related documentation#