How to automate configuration of clients and remote servers

Most processing in Deephaven installations takes place on core servers running roles like Data Import Server, Query Server, or Merge Server. The Deephaven installation and upgrade processes ensure that these machines have all needed binaries and configuration files so that processes running on them, such as loggers and tailers, can connect to infrastructural Deephaven services.

To connect other systems to a Deephaven installation, they require many of the same binaries and configuration files found on the core Deephaven servers. Client systems that run the Deephaven Console typically obtain these files through the Deephaven Launcher. Each time a connection is made to a Deephaven installation, the Launcher checks to ensure it has the latest files from the server and downloads any newer files as necessary.

Running remote services, such as loggers and tailers, is also common on machines not used for interactive consoles. For these systems, the Deephaven Updater can be used to script or run a command line update of needed components. This script is part of the Deephaven Launcher, and the whole process of installing the Launcher and using this class in "headless" installations is detailed below.

Note

See also Launcher and Client configuration

Find the Client Update Service URL

The Web API service manages the Client Update Service (CUS). This service runs on the "infra" node of a Deephaven cluster (as service web_api_service). The default ports are 8123 for the Web API service in general or 8443 for the CUS functionality. If the Deephaven system is configured to use Envoy, the default port is 8000.

The Deephaven Launcher can be downloaded from the base CUS URL, which will be one of the following if the default ports are used:

Starting with version 9.09, you can also download the Deephaven Launcher directly from JFrog:

The Deephaven Launcher or Updater will use the CUS to download all the Java and configuration files needed to interact with the Deephaven installation. The Deephaven instance is identified by a base URL called the appbase, which is defined in the file located at /etc/sysconfig/deephaven/illumon.d.latest/client_update_service/getdown.global. This address is based on the URL determined above, with the addition of /iris/.

Replace deephaven-infra-server.mycompany.net or deephaven-envoy-server.mycompany.net with the hostnames for the corresponding nodes in your Deephaven cluster.

If you have access to the Deephaven infra server, you can extract the correct URL from one of the CUS configuration files:

grep "appbase =" /etc/sysconfig/illumon.d/client_update_service/getdown.global

For Windows systems

  1. Download the Deephaven Launcher installer from the Client Update Service URL. This is a one-time install per system. Run the installer silently with the /S option, filling in the actual executable file name. For example:
deephaven-launcher-9.09.exe /S
  1. The Deephaven Launcher includes a .bat script to create and update a Deephaven instance. The script and program files will be in one of the following locations:
"%LocalAppData%\Programs\Illumon"
"%ProgramFiles(x86)%\Illumon"
"%ProgramFiles%\Illumon"

Run one of these commands in a cmd window to create instance "DR", filling in the correct path and Client Update Service URL for your environment:

"%LocalAppData%\Programs\Illumon\DeephavenUpdater.bat" DR <CUS_URL>/iris/
"%ProgramFiles(x86)%\Illumon\DeephavenUpdater.bat" DR <CUS_URL>/iris/

After the instance has been created, omit the URL to update the "DR" instance:

"%LocalAppData%\Programs\Illumon\DeephavenUpdater.bat" DR
"%ProgramFiles(x86)%\Illumon\DeephavenUpdater.bat" DR
  1. If a workspace path is needed for the class(es) being used (often required), create a workspace path to be passed in the command line when executing the client process.
mkdir %LocalAppData%\Illumon\DR\workspaces\default

The launch of a remote process accessing Deephaven will look similar to the following (note that the ^ is a line continuation character):

%LocalAppData%\Illumon\jdk\bin\javaw.exe ^
-classpath "%LocalAppData%\Illumon\DR\resources;^
%LocalAppData%\Illumon\DR\java_lib\*;^
[paths to any other resources]" ^
-server -Xmx256m ^
-DConfiguration.rootFile=iris-common.prop ^
-Dservice.name=iris_console ^
-Dworkspace=%LocalAppData%\Illumon\DR\workspaces\default ^
-Ddevroot=%LocalAppData%\Illumon\DR ^
-DLoggerFactory.teeOutput=true ^
-DAuthenticationClientManager.defaultPrivateKeyFile=[key file] ^
-Ddh.config.client.bootstrap=%LocalAppData%\Illumon\DR\tmp\dh-config\clients ^
[class to be run and any program arguments]

For Non-Windows systems

  1. Download the Deephaven Launcher .tar file from the Client Update Service URL. To install the program, extract it from the tar file. This installation is required only once per system. Navigate to an appropriate install location and extract the tar file. For example (fill in the actual file name):
cd [your_install_folder]

tar -xvf [download_path]/deephaven-launcher-9.09.tgz
  1. Create the instance and download client files:
[your_install_folder]/DeephavenUpdater.sh DR <CUS_URL>/iris/
  1. Check for updates and download client files, ideally at every launch of the client process:
[your_install_folder]/DeephavenUpdater.sh DR
  1. If a workspace path is needed for the class(es) being used (often required), create a path to be passed in the command line when executing classes. For example:
mkdir -p ~/iris/workspaces/DR/workspaces/default

The launch of a remote process accessing Deephaven will look similar to the following:

java -classpath "[instances_root]/DR/resources:\
[instances_root]/DR/java_lib/*:\
[paths to any other resources]" \
-server -Xmx256m \
-DConfiguration.rootFile=iris-common.prop \
-Dservice.name=iris_console \
-Dworkspace=/home/[username]/iris/workspaces/DR/workspaces/default \
-Ddevroot=[instances_root]/DR \
-DLoggerFactory.teeOutput=true \
-DAuthenticationClientManager.defaultPrivateKeyFile=[key file] \
-Ddh.config.client.bootstrap=[instances_root]\DR\tmp\dh-config\clients \
[class to be run and any program arguments]

Customizing locations

The Deephaven Updater process reads properties from several locations, where HOME is the current user's home directory, and INSTALLDIR is where the Deephaven Updater script is installed. Any properties passed to the process take precedence, followed by settings in the following order:

  • HOME/.illumon.launcher.properties
  • /etc/illumon.launcher.properties (non-Windows)
  • $PROGRAMDATA/illumon/illumon.launcher.properties (Windows)
  • INSTALLDIR/illumon/illumon.launcher.properties (Windows)
  • INSTALLDIR/lib/illumon.launcher.properties

Settings

You can change the default location for downloaded program files and workspace files by changing these properties in any file:

  • IrisConfigurationLauncher.instanceRoot=${user.home}/iris/.programfiles
  • IrisConfigurationLauncher.workspaceRoot=${user.home}/iris/workspaces

Additional properties and default values are documented in INSTALLDIR/lib/illumon.launcher.properties.

The DeephavenUpdater script has command line parameters for setting these locations.

Caution

Use these options with caution. If you change the locations on the command line, you must always include the same command line parameters, or the updater will not find previously downloaded program files.

Usage: DeephavenUpdater.sh <Instance name> [Server URL] [--instance-root=<directory>] [--workspace-root=<directory>] [--insecure]
    the optional Server URL argument will be used to create a new instance, if given.
    --instance-root sets the directory under which downloaded instance files will be stored.
    --workspace-root sets the directory under which instance workspace files will be stored.
    --insecure will print and trust the server certificates in order to download the initial files. THIS IS A SECURITY RISK.
For example:
    DeephavenUpdater.sh DeephavenProd
    DeephavenUpdater.sh DeephavenProd https://deephaven.prod.company.com:8443/iris
    DeephavenUpdater.sh DeephavenProd --instance-root=/tmp/instances --workspace-root=~/deephavenWorkspaces

Troubleshooting

The Deephaven Updater script will print the logging file to stdout. This file contains details about connections and certificates and should be sufficient to diagnose configuration issues. Deephaven support requires this information to help troubleshoot.

If the certificates used by the Deephaven server are unknown to your client, you can bypass the "trusted root" check by adding --insecure to the command line. This may allow the servers trust store files to be downloaded. These files will be used by the updater and the Classic console, and can be used by any client for TLS validation.

The following properties are useful to increase the information available (set them in any of the files named above):

  • IrisConfigurationLauncher.connectionTimeoutMs - Change the timeout used when attempting to connect to the server.
  • IrisConfigurationLauncher.verboseLogging=true - Verbose logging adds more detail about progress and full exception text.
  • javax.net.debug=all - Adds very verbose logging about certificates and TLS negotiations.