Build and launch from source
warning
This page is a guide for building and running Deephaven from source code. These instructions are for users and developers interested in tinkering with and modifying Deephaven source code.
Almost all users will want to Launch Deephaven from pre-built images. It is the easiest way to deploy.
If you are not sure which of the two is right for you, use the pre-built images. For detailed instructions on how to do this, see Launch Deephaven from pre-built images.
Deephaven Community Core is a real-time, time-series, column-oriented analytics engine with relational database features. Queries can seamlessly operate upon both historical and real-time data. It can ingest data from a variety of sources, apply computation and analysis algorithms to that data, and build rich queries, dashboards, and representations with the results.
In this tutorial, you'll learn how to set up and launch Deephaven.
Prerequisites
Required dependencies
Building and running Deephaven requires a few software packages.
Package | Version | OS |
---|---|---|
git | ^2.25.0 | All |
java | >=11 | All |
docker | ^20.10.8 | All |
docker-compose | ^1.29.0 | All |
Windows | 10 (OS build 20262 or higher) | Only Windows |
WSL | 2 | Only Windows |
You can check if these packages are installed and functioning by running:
git version
java -version
docker version
docker-compose version
docker run hello-world
note
Internally, the Java build process will use Gradle Auto Provisioning to download and use the appropriate Java version for building and testing.
On Windows, all commands must be run inside a WSL 2 terminal.
Installing WSL...
On Windows, Windows Subsystem for Linux (WSL) version 2 must be installed. WSL is not needed on other operating systems.
Instructions for installing WSL 2 can be found at https://docs.microsoft.com/en-us/windows/wsl/install-win10. The latest Ubuntu Linux distribution for WSL 2 is recommended.
Installing Java
Deephaven can be built with Oracle OpenJDK, Adoptium OpenJDK, Zulu OpenJDK, and likely other flavors of the OpenJDK.
You can choose to install with the vendor links above, or you may prefer to use your package manager's version, which is often the easiest route:
Debian / Ubuntu / Windows WSL2 Ubuntu
apt-get update
apt-get install openjdk-11-jdk-headlessFedora / RedHat
dnf install java-11-openjdk-devel
Mac
brew install [email protected]
OpenJDK 11 may need to be added to your path. For Intel Macs:
echo 'export PATH="/usr/local/opt/[email protected]/bin:$PATH"' >> ~/.zshrc
For M1 Macs:
echo 'export PATH="/opt/homebrew/opt/[email protected]/bin:$PATH"' >> ~/.zshrc
See Brew for more information.
Installing Docker
Instructions for installing and configuring Docker can be found at https://docs.docker.com/get-docker/. Windows users should follow the WSL2 instructions.
Instructions for installing and configuring docker-compose
can be found at
https://docs.docker.com/compose/install/.
Docker RAM settings
Tests run as part of the build process require at least 4GB of Docker RAM. To check your Docker configuration, run:
docker info | grep Memory
By default, Docker on Mac is configured with 2 GB of RAM. If you need to increase the memory on your Mac, click on the Docker icon on the top bar and navigate to Preferences->Resources->Memory
. Docker on Windows and Linux should not require configuration changes.
Docker WSL settings
On Windows, Docker must be configured to allow WSL to access Docker. In Docker Desktop, navigate to Settings->Resources->WSL Integration
, and enable your distribution. After restarting your WSL shell, you will be able to run Docker commands from WSL.
If docker run hello-world
does not work...
If docker run hello-world
does not work, try the following:
docker info
(Linux) Are you in the
docker
user group?sudo groupadd docker
sudo usermod -aG docker $USER
In order for the new group to apply, you must finish your current session and start a new one (log out and log in). Also, if the Gradle daemon is already running for your user, it needs to be stopped since it is running with the old credentials.
If `git clone` fails on WSL 2...
WSL 2 has a known bug that results in git clone
failures in some environments. The bug has been reported since 2019. You may be able to fix this as follows.
Update networking drivers:
- On Windows, open the Device Manager.
- Expand "Network adapters".
- Find which network device you are using (wifi, or wired), and note the brand.
- Google "newest Windows 10 device drivers for <brand_name>".
- Install the drivers.
- Restart.
Now you need to set the maximum network packet size, known as the maximum transmission unit (MTU), to something slightly smaller than the WSL interface value.
In powershell, lookup the current MTU for the WSL interface:
netsh interface ipv4 show subinterface
You will see output that looks like:
MTU MediaSenseState Bytes In Bytes Out Interface
------ --------------- --------- --------- -------------
4294967295 1 0 5150 Loopback Pseudo-Interface 1
1500 1 83773143 12179977 Wi-Fi
1500 5 0 0 Ethernet
1500 5 0 0 Local Area Connection* 1
1500 5 0 0 Local Area Connection* 2
1500 5 0 0 Ethernet 2
1500 1 29246 11502767 vEthernet (WSL)
Note the vEthernet interface's MTU.
In a WSL 2 Ubuntu shell, set the MTU to a number slightly smaller than the WSL vEthernet value obtained above. This ensures that there is enough buffer to wrap the packets:
sudo ip link set dev eth0 mtu 1350
Checkout and build Deephaven
Once all of the required dependencies are installed and functioning, run:
git clone https://github.com/deephaven/deephaven-core.git
cd deephaven-core
./gradlew prepareCompose
These commands will create:
- A
deephaven-core
directory containing the source code. - Docker images containing everything needed to launch Deephaven.
Run Deephaven
note
Commands in the following sections for interacting with a deployment must be run from the project's root directory, deephaven-core
.
important
On an M1 Mac, Deephaven may fail to start if one of the public images
was used previously. Stop and remove all Docker containers, then run docker system prune -a
to remove the image.
Python
From the deephaven-core
directory, run
docker-compose up
This will start Deephaven. The console will fill with status and logging output.
Killing the process (e.g. Ctrl+C
) will stop Deephaven.
Groovy
From the deephaven-core
directory, run
docker-compose --env-file default_groovy.env up
This will start Deephaven. The console will fill with status and logging output.
Killing the process (e.g., Ctrl+C
) will stop Deephaven.
Run Deephaven IDE
Once Deephaven is running, you can launch a Deephaven IDE in your web browser. Deephaven IDE allows you to interactively analyze data and develop new analytics.
- If Deephaven is running locally, navigate to http://localhost:10000/ide/.
- If Deephaven is running remotely, navigate to
http://<hostname>:10000/ide/
, where<hostname>
is the address of the machine Deephaven is running on.