Build and launch from source
This page is a guide for building and running Deephaven from source code. These instructions are for users and developers interested in tinkering with and modifying Deephaven source code.
The majority of users will want to Launch Deephaven from pre-built images. It is the easiest way to deploy.
If you are not sure which of the two is right for you, use the pre-built images. For detailed instructions on how to do this, see Launch Deephaven from pre-built images.
Deephaven Community Core is a real-time, time-series, column-oriented analytics engine with relational database features. Queries can seamlessly operate upon both historical and real-time data. It can ingest data from a variety of sources, apply computation and analysis algorithms to that data, and build rich queries, dashboards, and representations with the results.
In this tutorial, you'll learn how to set up and launch Deephaven.
Prerequisites
Required dependencies
Building and running Deephaven requires a few software packages.
Package | Version | OS |
---|---|---|
git | ^2.25.0 | All |
java | >=11, <20 | All |
docker | ^20.10.8 | All |
Windows | 10 (OS build 20262 or higher) | Only Windows |
WSL | 2 | Only Linux via Windows |
You can check if these packages are installed and functioning by running:
git version
java -version
docker version
docker compose version
docker run hello-world
Internally, the Java build process will use Gradle Auto Provisioning to download and use the appropriate Java version for building and testing.
Installing WSL...
Deephaven can be run natively on Windows, without installing WSL. However, users who want to run Deephaven inside a GNU/Linux environment on a Windows machine will need Windows Subsystem for Linux (WSL) version 2. WSL is not needed on other operating systems.
Instructions for installing WSL 2 can be found at https://docs.microsoft.com/en-us/windows/wsl/install-win10. The latest Ubuntu Linux distribution for WSL 2 is recommended.
Installing Java
Deephaven can be built with Oracle OpenJDK, Adoptium OpenJDK, Zulu OpenJDK, and likely other flavors of the OpenJDK.
You can choose to install with the vendor links above, or you may prefer to use your package manager's version, which is often the easiest route:
Debian / Ubuntu / Windows WSL2 Ubuntu
apt-get update
apt-get install openjdk-11-jdk-headlessFedora / RedHat
dnf install java-11-openjdk-devel
Mac
brew install openjdk@11
OpenJDK 11 may need to be added to your path. For Intel Macs:
echo 'export PATH="/usr/local/opt/openjdk@11/bin:$PATH"' >> ~/.zshrc
For M1 Macs:
echo 'export PATH="/opt/homebrew/opt/openjdk@11/bin:$PATH"' >> ~/.zshrc
See Brew for more information.
Installing Docker
Instructions for installing and configuring Docker can be found at https://docs.docker.com/get-docker/. Windows users should follow the WSL2 instructions.
Instructions for installing and configuring docker compose
can be found at
https://docs.docker.com/compose/install/.
Docker RAM settings
Tests run as part of the build process require at least 4GB of Docker RAM. To check your Docker configuration, run:
docker info | grep Memory
By default, Docker on Mac is configured with 2 GB of RAM. If you need to increase the memory on your Mac, click on the Docker icon on the top bar and navigate to Preferences->Resources->Memory
. Docker on Windows and Linux should not require configuration changes.
Docker WSL settings
If using WSL, you will need to change Docker's configuration settings to allow WSL access. In Docker Desktop, navigate to Settings->Resources->WSL Integration
, and enable your distribution. After restarting your WSL shell, you will be able to run Docker commands from WSL.
If docker run hello-world
does not work...
If docker run hello-world
does not work, try the following:
docker info
(Linux) Are you in the
docker
user group?sudo groupadd docker
sudo usermod -aG docker $USER
In order for the new group to apply, you must finish your current session and start a new one (log out and log in). Also, if the Gradle daemon is already running for your user, it needs to be stopped since it is running with the old credentials.
If `git clone` fails on WSL 2...
WSL 2 has a known bug that results in git clone
failures in some environments. The bug has been reported since 2019. You may be able to fix this as follows.
Update networking drivers:
- On Windows, open the Device Manager.
- Expand "Network adapters".
- Find which network device you are using (wifi, or wired), and note the brand.
- Google "newest Windows 10 device drivers for <brand_name>".
- Install the drivers.
- Restart.
Now you need to set the maximum network packet size, known as the maximum transmission unit (MTU), to something slightly smaller than the WSL interface value.
In powershell, lookup the current MTU for the WSL interface:
netsh interface ipv4 show subinterface
You will see output that looks like:
MTU MediaSenseState Bytes In Bytes Out Interface
------ --------------- --------- --------- -------------
4294967295 1 0 5150 Loopback Pseudo-Interface 1
1500 1 83773143 12179977 Wi-Fi
1500 5 0 0 Ethernet
1500 5 0 0 Local Area Connection* 1
1500 5 0 0 Local Area Connection* 2
1500 5 0 0 Ethernet 2
1500 1 29246 11502767 vEthernet (WSL)
Note the vEthernet interface's MTU.
In a WSL 2 Ubuntu shell, set the MTU to a number slightly smaller than the WSL vEthernet value obtained above. This ensures that there is enough buffer to wrap the packets:
sudo ip link set dev eth0 mtu 1350
Build and run Deephaven
The following instructions are a condensed version of instructions found in the deephaven-core repository. For the full instructions with explanations of configuration parameters, SSL, and more, see the README.
The following steps show how to build and run Deephaven with Python from source. For Groovy, see here, or click the Language
drop-down menu in the top left of the webpage, and select Java (Groovy)
.
Clone the deephaven-core repository
Once all of the required dependencies are installed and functioning, run:
git clone https://github.com/deephaven/deephaven-core.git
cd deephaven-core
The first of these lines clones the repository; the second line sets the directory to the new local clone.
Set up the Python virtual environment
First, set up a virtual environment.
python -m venv /tmp/my-dh-venv
source /tmp/my-dh-venv/bin/activate
Build and install the wheel
Then, build and install the wheel.
./gradlew py-server:assemble
pip install "py/server/build/wheel/deephaven_core-<version>-py3-none-any.whl[autocomplete]"
Where <version>
is replaced with the Gradle version in use:
./gradlew -q printVersion
If you prefer not to use autocomplete, remove [autocomplete]
from the wheel installation.
Build and run
Lastly, build and run Deephaven.
./gradlew server-jetty-app:run
Run Deephaven IDE
Once Deephaven is running, you can launch a Deephaven IDE in your web browser. Deephaven IDE allows you to interactively analyze data and develop new analytics.
- If Deephaven is running locally, navigate to http://localhost:10000/ide/.
- If Deephaven is running remotely, navigate to
http://<hostname>:10000/ide/
, where<hostname>
is the address of the machine Deephaven is running on.
Authentication
Deephaven, by default, uses pre-shared key authentication. If no key is set, a randomly generated key will be used to log into the server each time it starts. The randomly generated key is printed to the Docker logs like this:
To set your own pre-shared key, add -Ppsk=<YourPasswordHere>
:
./gradlew server-jetty-app:run -Ppsk=DeephavenRocks!