How to automate configuration of clients and remote servers

Most processing in Deephaven installations takes place on core servers running roles like Data Import Server, Query Server, or Merge Server. The Deephaven installation and upgrade processes ensure that these machines have all needed binaries and configuration files so that processes running on them, such as loggers and tailers, can connect to infrastructural Deephaven services.

For other systems to connect to a Deephaven installation, they need most of the same binaries and configuration files that are installed on the core Deephaven servers. Client systems running the Deephaven Console typically get these files with the Deephaven Launcher. At each connection to a Deephaven installation, the Launcher checks to verify that it has the latest files from the server, and downloads newer files as needed.

It is also common to run remote services, such as loggers and tailers, on machines that are not used for interactive consoles. For these systems, the Deephaven Updater can be used to script or run a command line update of needed components. This script is part of the Deephaven Launcher, and the whole process of installing the Launcher and using this class in "headless" installations is detailed below.

Find the Client Update Service URL

The Client Update Service runs on the "infra" node of a Deephaven cluster (as service client_update_service). The default port is 8443. If the Deephaven system is configured to use Envoy, the default port is 8000.

The Deephaven Launcher can be downloaded from the base Client Update Service URL, which will be one of the following if the default ports are used:

The Deephaven Launcher or Updater will use the Client Update Service to download all the Java and configuration files needed to interact with the Deephaven installation. The Deephaven instance is identified by a base URL called the appbase. This address is based on the URL determined above, with the addition of /iris/.

If you have access to the Deephaven infra server, you can extract the correct URL from one of the CUS configuration files:

grep "appbase =" /etc/sysconfig/illumon.d/client_update_service/getdown.global

For Windows Systems

  1. Download the Deephaven Launcher installer from the Client Update Service URL. There are two options: one with and without a JDK. This is a one-time install per system. Run the installer silently with the /S option, filling in the actual executable file name. For example:
DeephavenLauncherSetupWithJdk1.8.0_162_9.02.exe /S
  1. The Deephaven Launcher includes a .bat script to create and update a Deephaven instance. The script and program files will be in one of the following locations:
"%LocalAppData%\Programs\Illumon"
"%ProgramFiles(x86)%\Illumon"
"%ProgramFiles%\Illumon"

Run one of these commands in a cmd window to create instance "DR", filling in the correct path and Client Update Service URL for your environment:

"%LocalAppData%\Programs\Illumon\DeephavenUpdater.bat" DR <CUS_URL>/iris/
"%ProgramFiles(x86)%\Illumon\DeephavenUpdater.bat" DR <CUS_URL>/iris/

After the instance has been created, omit the URL to update the "DR" instance:

"%LocalAppData%\Programs\Illumon\DeephavenUpdater.bat" DR
"%ProgramFiles(x86)%\Illumon\DeephavenUpdater.bat" DR
  1. If a workspace path is needed for the class(es) being used (often required), create a workspace path to be passed in the command line when executing the client process.
mkdir %LocalAppData%\Illumon\DR\workspaces\default

The launch of a remote process accessing Deephaven will look similar to the following (note that the ^ is a line continuation character):

%LocalAppData%\Illumon\jdk\bin\javaw.exe ^
-classpath "%LocalAppData%\Illumon\DR\resources;^
%LocalAppData%\Illumon\DR\java_lib\*;^
[paths to any other resources]" ^
-server -Xmx256m ^
-DConfiguration.rootFile=iris-common.prop ^
-Dservice.name=iris_console ^
-Dworkspace=%LocalAppData%\Illumon\DR\workspaces\default ^
-Ddevroot=%LocalAppData%\Illumon\DR ^
-DLoggerFactory.teeOutput=true ^
-DAuthenticationClientManager.defaultPrivateKeyFile=[key file] ^
-Ddh.config.client.bootstrap=%LocalAppData%\Illumon\DR\tmp\dh-config\clients ^
[class to be run and any program arguments]

For Non-Windows Systems

  1. Download the Deephaven Launcher .tar file from the Client Update Service URL. Install the program by extracting it from the tar file. This is a one-time install per system. Navigate to an appropriate install location and extract the tar file. For example (fill in the actual file name):
cd [your_install_folder]

tar -xvf [download_path]/DeephavenLauncher_9.02.tar
  1. Create the instance and download client files:
[your_install_folder]/DeephavenUpdater.sh DR <CUS_URL>/iris/
  1. Check for updates and download client files, ideally at every launch of the client process:
[your_install_folder]/DeephavenUpdater.sh DR
  1. If a workspace path is needed for the class(es) being used (often required), create a path to be passed in the command line when executing classes. For example:
mkdir -p ~/iris/workspaces/DR/workspaces/default

The launch of a remote process accessing Deephaven will look similar to the following:

java -classpath "[instances_root]/DR/resources:\
[instances_root]/DR/java_lib/*:\
[paths to any other resources]" \
-server -Xmx256m \
-DConfiguration.rootFile=iris-common.prop \
-Dservice.name=iris_console \
-Dworkspace=/home/[username]/iris/workspaces/DR/workspaces/default \
-Ddevroot=[instances_root]/DR \
-DLoggerFactory.teeOutput=true \
-DAuthenticationClientManager.defaultPrivateKeyFile=[key file] \
-Ddh.config.client.bootstrap=[instances_root]\DR\tmp\dh-config\clients \
[class to be run and any program arguments]

Customizing locations

The Deephaven Updater process reads properties from several locations, where HOME is the current user's home directory, and INSTALLDIR is where the Deephaven Updater script is installed. Any properties passed to the process take precedence, followed by settings in the following order:

  • HOME/.illumon.launcher.properties
  • /etc/illumon.launcher.properties (non-Windows)
  • $PROGRAMDATA/illumon/illumon.launcher.properties (Windows)
  • INSTALLDIR/illumon/illumon.launcher.properties (Windows)
  • INSTALLDIR/lib/illumon.launcher.properties

Settings

You can change the default location for downloaded program files and workspace files by changing these properties in any file:

  • IrisConfigurationLauncher.instanceRoot=${user.home}/iris/.programfiles
  • IrisConfigurationLauncher.workspaceRoot=${user.home}/iris/workspaces

Additional properties and default values are documented in INSTALLDIR/lib/illumon.launcher.properties.

The DeephavenUpdater script has command line parameters for setting these locations.

Caution

Use these options with caution - if you change the locations on the command line, you must always include the same command line parameters, or the updater won't find previously downloaded program files.

Usage: DeephavenUpdater.sh <Instance name> [Server URL] [--instance-root=<directory>] [--workspace-root=<directory>] [--insecure]
    the optional Server URL argument will be used to create a new instance, if given.
    --instance-root sets the directory under which downloaded instance files will be stored.
    --workspace-root sets the directory under which instance workspace files will be stored.
    --insecure will print and trust the server certificates in order to download the initial files. THIS IS A SECURITY RISK.
For example:
    DeephavenUpdater.sh DeephavenProd
    DeephavenUpdater.sh DeephavenProd https://deephaven.prod.company.com:8443/iris
    DeephavenUpdater.sh DeephavenProd --instance-root=/tmp/instances --workspace-root=~/deephavenWorkspaces

Troubleshooting

The Deephaven Updater script will print the logging file to STDOUT. This file will contain the details about connections and certificates, and should be sufficient to diagnose configuration issues. Deephaven support will need this information to help troubleshoot.

If the certificates used by the Deephaven server are unknown to your client, you can bypass the "trusted root" check by adding --insecure to the command line. This may allow the servers trust store files to be downloaded. These files will be used by the updater and the Classic console, and can be used by any client for TLS validation.

The following properties are useful to increase the information available (set them in any of the files named above):

  • IrisConfigurationLauncher.connectionTimeoutMs - Change the timeout used when attempting to connect to the server.
  • IrisConfigurationLauncher.verboseLogging=true - Verbose logging adds more detail about progress and full exception text.
  • javax.net.debug=all - Adds very verbose logging about certificates and TLS negotiations.