Configure the Deephaven native application

This documentation provides low-level details about the Deephaven native application bootstrap configuration and startup process. Users who wish to get up and running without needing to worry about these details may prefer to check out the Docker application documentation.

Native Application Script

One of the primary artifacts that Deephaven produces is a native application, available in both tar and zip formats. The latest application artifacts can be downloaded from our releases page. The native application is currently targeted at Unix-like systems: Linux or macOS¹.

The inner structure of the application is a root directory of the form server-jetty-<version>/ with further subdirectories bin/, lib/, and etc/². The bin/ directory contains a start script, the lib/ directory contains all of the JAR files that form the classpath for the application, and the etc/ directory contains extra files that may be useful for configuring the JVM.

The start script contains the logic to collect the arguments, assemble the classpath, and execute the java command. There are several environment variables that influence the start script:

Environment Variable	Description	Default Value	Example
JAVA_OPTS	The main JVM arguments. These are generated by the start script, but can be overridden.	Deephaven-recommended arguments and property definitions.	N/A. It is not recommended to set JAVA_OPTS.
START_OPTS	Additional user-supplied JVM arguments. These are added to the command line after JAVA_OPTS.	None	START_OPTS="-Xms4g -Xmx4g -Dmyproperty=myvalue -Ddeephaven.console.type=groovy"
EXTRA_CLASSPATH	Additional directories to include in the Java classpath (e.g., /apps/libs/*)	None	EXTRA_CLASSPATH="/apps/libs/*:/opt/my_java_libs/*

The end result of the start script is a command that looks something like:

java <required-opts> $JAVA_OPTS $START_OPTS -classpath <classpath> io.deephaven.server.jetty.JettyMain [optional-bootstrap-file]

• <required-opts> are the pre-defined JVM arguments that Deephaven requires. These may include --add-export or --add-opens arguments, compiler directives, and others. The user will not be able to change these.
• JAVA_OPTS and START_OPTS are JVM arguments, as described above.
• <classpath> is the constructed classpath. The user cannot directly change this classpath; however, if the EXTRA_CLASSPATH environment variable is set, it will be prepended to the constructed classpath.
• [optional-bootstrap-file] is an optional file argument the user may provide to the start script. When provided, the properties in this file will take precedent over any other system properties. Often, it is better to use START_OPTS or a configuration file to edit properties. This option may be removed in a future release.

For example, to launch Deephaven natively with a Python console, users can run:

./server-jetty-<version>/bin/start

To launch Deephaven natively with a Groovy console, including additional Java libraries from the /opt/mylibs directory, set the EXTRA_CLASSPATH variable:

EXTRA_CLASSPATH="/opt/mylibs/*" START_OPTS="-Ddeephaven.console.type=groovy" ./server-jetty-<version>/bin/start

Note: Python execution, configured by default, requires a proper Python environment. See the Python environment section for more details.

Deephaven server bootstrap configuration

The Deephaven server process has a few essential points of bootstrapping configuration that the user may choose to set. Note, bootstrapping configuration is limited in scope, and is a prerequisite to the Deephaven configuration file.

Application name

The Deephaven application name is used to inform default values for other pieces of configuration. On systems that run multiple native Deephaven server processes, it may be useful to give each application a unique name.

The default application name is deephaven.

The user may explicitly set this value via the system property deephaven.application or the environment variable DEEPHAVEN_APPLICATION.

Data directory

The data directory is a directory where users and their applications may read or write data. Deephaven services may also use this directory for storage purposes. For example, the Deephaven storage service uses <dataDir>/storage by default for its storage needs.

The data directory is based on OS-conventions by default. On Linux, this is $HOME/.local/share/<application>/, and on macOS this is $HOME/Library/Application Support/io.Deephaven-Data-Labs.<application>/.

The user may explicitly set this value via the system property deephaven.dataDir or the environment variable DEEPHAVEN_DATA_DIR.

Cache directory

The cache directory is a directory where the Deephaven server may read and write cache data.

The cache directory is based on OS-conventions by default. On Linux, this is $HOME/.cache/<application>/, and on macOS this is $HOME/Library/Caches/io.Deephaven-Data-Labs.<application>/.

The user may explicitly set this value via the system property deephaven.cacheDir or the environment variable DEEPHAVEN_CACHE_DIR.

Config directory

The config directory is a directory where users may store application configuration information. On startup, the Deephaven server will load the Deephaven configuration file <configDir>/deephaven.prop if it exists, and otherwise fallback to the built-in dh-defaults.prop configuration file. Other Deephaven services may key off of the <configDir>/ in the future.

The config directory is based on OS-conventions by default. On Linux, this is $HOME/.config/<application>/, and on macOS this is $HOME/Library/Application Support/io.Deephaven-Data-Labs.<application>/.

The user may explicitly set this value via the system property deephaven.configDir or the environment variable DEEPHAVEN_CONFIG_DIR.

For more information, see the configuration file guide.

Quiet flag

The quiet flag control bootstrap logging written to stdout on startup. When true, bootstrap logging will not be output.

The quiet flag is false by default.

The user may explicitly set this value via the system property deephaven.quiet or the environment variable DEEPHAVEN_QUIET.

Python environment

A Python environment with the deephaven-core PyPi package installed is necessary to start up the Deephaven native application with a Python session. It is recommended to use a virtual environment. deephaven-core has an optional "autocomplete" feature.

# Setup python virtual environment
python3 -m venv /path/to/deephaven-venv
source /path/to/deephaven-venv/bin/activate

# Note: the optional "autocomplete" feature may be left out if web UI autocomplete is not needed
pip install deephaven-core[autocomplete]==<version>

# Start Deephaven with python session
START_OPTS="-Ddeephaven.console.type=python" ./server-jetty-<version>/bin/start

Sourcing the Python virtual environment activate script has the side effect of setting the VIRTUAL_ENV environment variable, and this is what the Deephaven server keys off of.

If you have multiple Deephaven servers on the same machine with different Python requirements, it may be best to be explicit about which Deephaven server uses which virtual environment:

# Start "foo" with the virtual environment at /path/to/foo-venv
DEEPHAVEN_APPLICATION=foo VIRTUAL_ENV=/path/to/foo-venv ./server-jetty-<version>/bin/start

# Start "bar" with the virtual environment at /path/to/bar-venv
DEEPHAVEN_APPLICATION=bar VIRTUAL_ENV=/path/to/bar-venv ./server-jetty-<version>/bin/start

Downstream

Docker images

The Deephaven Docker images are based on the native application as described above, but with Docker specific bootstrap values. For example, the images set the environment variables DEEPHAVEN_DATA_DIR=/data, DEEPHAVEN_CACHE_DIR=/cache, and DEEPHAVEN_CONFIG_DIR=/opt/deephaven/config.

For more information, see the Docker application documentation.

Python embedded server

The Deephaven Python embedded server is a PyPi package that includes all the logic to start up the the Deephaven server natively from within python. It does not use the start script as described above, but the Deephaven server bootstrap configuration logic still applies.

Related documentation

• How to configure the Docker application
• How to create a Deephaven configuration file
• Install and run from pre-built artifacts
• Build from source

---

1. Windows native support may be possible in the future.↩
2. For those interested in the nitty gritty, check out the Gradle application plugin.↩