Deephaven Launcher and Client configuration

Client Components Update

When a new Deephaven console is launched from the Launcher, the Launcher will check whether the instance has the latest versions of JAR files and other files from the Client Update Service (CUS) associated with the instance. It will download newer files if updates are available. Once the console is running, it will check for changes every 10 minutes. If new files are found (because an update has been applied or a configuration file changed on the server), a message banner will appear on the client console to inform the user that the console is out of date. A relaunch of the console will then be required to download and apply the new files. Continuing to run the console with out of date components may appear to work fine. However, problems may arise when client-side settings are not correct for the current environment, or when the client is using classes that no longer match the versions installed on the server.

When the Web API Service is started or a reload request is received, a temporary directory (e.g., /tmp/cus7948278661189215940/) is populated, syncing files from system directories and configuration. A digest file is created that the launcher uses to avoid re-downloading unchanged files. The files in this temporary directory should never be edited.

The getdown.txt file on the server inside this temporary directory controls most of the Client Update Service behavior. This file is dynamically built by the Client Update Service when it starts and when it reloads, using files from the directory /etc/sysconfig/illumon.d/client_update_service, and adding required resources from the Deephaven installation. This directory is also recreated each time the Deephaven software is installed, but existing files are copied to the new directory.

getdown.txt is constructed by concatenating two files:

• a system-owned /etc/sysconfig/illumon.d/client_update_service/iris/iris/getdown.txt.pre, which is automatically replaced every time Deephaven is upgraded.
• a customer-owned /etc/sysconfig/illumon.d/client_update_service/getdown.global, which is where all customer-controlled configuration should be placed. This file is never replaced. The exception is the appbase configuration, which is set whenever the Client Update Service is reloaded.

For a comprehensive explanation of system vs. customer files, see the Client Update Service documentation.

The customer-owned files are as follows:

• getdown.global - this contains all the parameters used by getdown and contains most of the parameters copied to getdown.txt.
• getdown.java.11, getdown.java.17 - these files contain default JVM parameters that will be use by clients based on the Java installation used.

When any of the files needed by the client are updated, you must reload the Client Update Service to regenerate the configuration files and digest. The CUS is restarted automatically after a Deephaven product RPM is applied to a query server. The updated digest2.txt will allow running clients to see that updates are available when they next check for changes.

Customer-Updateable files

The main settings and sections in getdown.global and their purposes follow.

appbase

This indicates the URL from which all subsequent files will be downloaded. All other files are relative to this root. This is generated from protocol of the Web API Service (http vs https), and the configurable host and port properties (client_update_service.host and client_update_service.port). Generally, it should not be directly edited. If the system is configured to use Envoy, then appbase will be the URL to the Envoy proxy server. In most cases, this will have been set by enabling Envoy configuration in the cluster.cnf file used by the installer. If Envoy was configured manually rather than through the installer, then it will be necessary to set the Envoy URL for the appbase value in getdown.global. Whether configured manually or automatically, the appbase value for Envoy will differ from the client_update_service.port; however, these values are still used for routing communications from Envoy to the Web API service that hosts the Client Update Service files.

appbase = https://irisprod.company.com/iris

To change the appbase in non-Envoy clusters, you should change the client_update_service.host and client_update_service.port properties, as detailed in the Client Update Service documentation.

The default page (index.html) for the parent of this path contains overview information about the Deephaven client and provides installer downloads. This page can be customized with environment-specific information, such as whom to contact for access to Deephaven.

ui.install_error

Certain errors will point to this page for more information. The default page provides contact information for Deephaven Data Labs support. It can be customized to add internal support information if desired. To customize, create a new file inside /etc/sysconfig/illumon.d/client_update_service/iris/ and set the client_update_service.install.error.file configuration property to this file. Then, reload the Client Update Service. Note that the value of ui.install_error is generated on CUS reload or restart, and should not be edited directly to avoid losing your changes. This value is handled like the appbase value, in that, in Envoy clusters, it should direct to the URL of the Envoy server. To change this value in non-Envoy environments, follow the instructions in the Client Update Service documentation.

ui.install_error = https://irisprod.company.com/iris/error.html

ui.name

The default value is ui.name = Deephaven.

However, users can customize this setting. For example:

ui.name = Deephaven Data Labs for Company Name Here

This will be the title of the application while downloading files.

Configuration.rootFile

This sets the properties file to be used by the client. The default is iris-common.prop.

jvmarg = -DConfiguration.rootFile=iris-common.prop

server_java_version

The Deephaven Launcher reads the property server_java_version = xx from getdown.txt, and will require that Deephaven be started with a matching Java version, where matching means that the version from java -version starts with xx. The required version comes from /usr/illumon/latest/etc/JAVA_VERSION, and the property values come from /etc/sysconfig/illumon.d/client_update_service/getdown.java.xx. This functionality requires Launcher 8.14 or higher, and appropriate settings on the server. The property can be added manually to the getdown.java.xx file on the server to enable this feature.

The Launcher will try the version used to run the Launcher first. If that doesn’t match, it will present dialogs allowing the user to select a new JDK. One option is to continue with an "invalid" version, primarily to handle future unforeseen situations.

Non-Updateable files

The main settings and sections in getdown.txt.pre and their purposes follow. These cannot be edited.

classpath

Resources will be searched for in the order these directories are listed, including searching inside JARs. Most Deephaven files are in the resources and java_lib directories. There are separate settings for classpaths on Windows, Linux, and OSX.

jvmarg = -classpath

jvmarg = [windows] "%APPDIR%\private_classes;%APPDIR%\private_jars\*;%APPDIR%\override;%APPDIR%\hotfixes\*;%APPDIR%\config;%APPDIR%\resources;%APPDIR%\java_lib\*;%APPDIR%\plugins\hotfixes\*;%APPDIR%\plugins\override;%APPDIR%\plugins;%APPDIR%\plugins\*;%APPDIR%\customer_jars\*"

jvmarg = [linux] %APPDIR%/private_classes:%APPDIR%/private_jars/*:%APPDIR%/override:%APPDIR%/hotfixes/*:%APPDIR%/config:%APPDIR%/resources:%APPDIR%/java_lib/*:%APPDIR%/plugins/hotfixes/*:%APPDIR%/plugins/override:%APPDIR%/plugins:%APPDIR%/plugins/*:%APPDIR%/customer_jars/*

jvmarg = [mac] %APPDIR%/private_classes:%APPDIR%/private_jars/*:%APPDIR%/override:%APPDIR%/hotfixes/*:%APPDIR%/config:%APPDIR%/resources:%APPDIR%/java_lib/*:%APPDIR%/plugins/hotfixes/*:%APPDIR%/plugins/override:%APPDIR%/plugins:%APPDIR%/plugins/*:%APPDIR%/customer_jars/*

%APPDIR%/config/*:%APPDIR%/private_classes:%APPDIR%/private_jars/*:%APPDIR%/override/*:%APPDIR%/resources:%APPDIR%/java_lib/*

clean_target and clean_target_resource

These specify directories under the instance directory whose contents should be completely replaced when updating. This is required for parts of the classpath that use wildcards to find JARs whose names will change from version to version, and to handle resources that are removed. Any files in directories named by a clean_target or clean_target_resource line will be removed unless they are named on a code = or resource = line. Note: Deephaven Launchers with version <= 8.15 do not look at resource = when handling clean_target, and do not process clean_target_resource. Directories that might contain resource = resources should use clean_target_resource.

clean_target = java_lib
<numerous clean_target entries follow>

clean_target_resource = calendars
clean_target_resource = chartThemes

While it is possible to add additional values for clean_target and clean_target_resource to getdown.global, in practice, it is very rare to need to do so. Do NOT edit targets in getdown.txt.pre, as they will be lost upon upgrading.

Code and resources

Code and resource entries in getdown.txt identify specific files for which digest values should be calculated, and that the Launcher should pull down to a client. The base path for these files is the system's temporary directory, e.g., /tmp/cus7948278661189215940/. The reload function of the Client Update Service creates symbolic links to (or, in some cases, directly copies) resources in several server directories before regenerating getdown.txt and updating the digest. The default sources are:

• /usr/illumon/latest/java_lib/
• /etc/sysconfig/illumon.d/java_lib/
• /etc/sysconfig/illumon.d/override/
• /etc/sysconfig/illumon.d/hotfixes/
• /etc/sysconfig/illumon.d/integrations/
• /etc/sysconfig/illumon.d/resources/
• /etc/sysconfig/illumon.d/calendars/
• /etc/sysconfig/illumon.d/chartthemes/
• /etc/sysconfig/illumon.d/dh-config/

Certain items are excluded, such as configuration files not needed by client installations. It will also include all the properties files stored in etcd; if you update your properties in etcd, you will need to reload the Client Update Service to have those new files sent to client machines.

Additional files Deephaven clients need may be added to /etc/sysconfig/illumon.d/java_lib/ or /etc/sysconfig/illumon.d/resources/. These files will also be included in the classpath of server processes. Individual code entries can be added to getdown.global to make files available to clients.

Like many other text configuration files, getdown.global, getdown.txt, and getdown.txt.pre use the pound/hash symbol (#) to indicate comment lines.

Uploading custom scripts and resources to Swing Clients

It may be required to execute a custom script on the swing clients, for example to download jars from a repository. Other resources can also be uploaded to clients. This can be done as follows.

The directory /etc/sysconfig/illumon.d/client_update_service/resources contains additional files which will be copied to the getdown directory. All this should be done as the Deephaven administrative user, typically irisadmin.

1. Create your additional files in the directory. For example, to create a script to be executed on the swing clients two files will be needed. The first is the script, which in this example is called prelaunch.sh:

   #!/usr/bin/env bash
   <Script commands>
   

2. The second must be called prelaunch.txt, and contains commands to be executed by the uploading client. A shell command is specified with exec. getdown won't create the script with executable permissions, so this must be done as a command. In this example:

   exec = chmod 754 ./prelaunch.sh
   exec = ./prelaunch.sh
   

3. Update /etc/sysconfig/illumon.d/client_update_service/getdown.global to include the new resources.

   # prelaunch file
   resource = prelaunch.txt
   resource = prelaunch.sh
   

4. Reload the CUS.

Launcher properties files

Properties files and JAR files are downloaded automatically from the Deephaven server by the Launcher, so no persistent changes can be made by editing these files on the client. To make global changes, make the changes on the server and reload the Client Update Service.

To apply configuration changes to the properties of specific client installations, these can be made by adding property keys and values to one of the jvmargs.txt files. There are several copies of this file that have varying levels of applicability depending on the location:

• <WorkspaceRoot>/jvmargs.txt - applies to all instances
• <WorkspaceRoot>/<instance_name>/jvmargs.txt - applies to all workspaces and consoles in the instance
• <WorkspaceRoot>/<instance_name>/workspaces/<workspace_name>/jvmargs.txt - applies to this Deephaven workspace
• <WorkspaceRoot>/<instance_name>/workspaces/console/jvmargs.txt - applies to consoles in the instance

Settings from all applicable files will be combined. If there are settings that exist in more than one of the files, the last one evaluated will take precedence. They are evaluated in the order listed above. The settings in the jvmargs.txt files also override settings from properties files.

The format of jvmargs.txt is similar to other properties files in that blank lines are ignored and comments start with a pound/hash (#) symbol. Because this file is being used to pass extra arguments into the Java command line, most of its key=value entries will start with -D. For example:

-DCsvExport.useFileSelectionDialog=true

Client Update Service Logs

The CUS is hosted by the Web API Service and will log to the same files inside /var/log/deephaven/web_api_service. These logs can be used to diagnose issues or be sent to Deephaven support.

Installation Location Information

Three types of files are stored during Deephaven Launcher installation and operation:

• Deephaven Launcher program files - These are the files required to run the Deephaven Launcher. These files do not change, and they do not contain data, downloaded JARs, or workspace files. On Windows clients, these files are installed on a per-user basis or per-machine basis, depending on the installation option chosen.
• Program files (also known as instance files) - These files include the JARs and the configuration files downloaded and synchronized for each instance, as well as log files generated by the Deephaven Launcher and Console programs.
• Workspace files - These files are workspace settings, backups, and history files.

The Deephaven Launcher displays the locations of Workspaces, Program, and Log files at the bottom right, as shown here:

Deephaven Launcher program files

Whether or not the Deephaven Installer is used for the installation, the program will be started from a working directory containing the shell script or executable, and supporting JAR and resource files.

Program files

A Deephaven instance specifies the server and port to which Deephaven should connect for that session. Users can configure multiple instances in the Deephaven Launcher.

The <programfiles>/<instance>/getdown.txt file enables an instance to appear as a choice in the Launcher. For example, if programfiles/foo/getdown.txt exists, then "foo" will be listed as an instance.

By default, program files and log files are stored under the following locations:

• Windows: %LOCALAPPDATA%/illumon
• macOS and Linux: ~/iris/.programfiles

Workspace files

Multiple workspaces can be created for each Deephaven instance. In the Launcher, the Workspace drop-down menu will list all available Workspaces for the selected Instance.

By default, workspace files are stored under the following locations:

• Windows: ~/Documents/Iris
• macOS and Linux: ~/iris/workspaces

For example, instance "foo" and workspace "Default" will be in ~/iris/workspaces/foo/workspaces/Default.

Deephaven Launcher Properties files

When the Launcher starts, it reads properties from all the launcher properties files it can find. Property values will be updated to replace a leading ~ or ${user.home} with the user's home directory. Files will be loaded in the order shown below. Later property settings override earlier ones, and system properties always have precedence.

Deephaven Launcher program properties file

The Launcher properties file (illumon.launcher.properties) is located in the Deephaven Launcher program folder.

System-wide properties file

The system-wide properties file (illumon.launcher.properties) is located in the Deephaven Launcher program folder:

• Windows: %PROGRAMDATA%/illumon/illumon.launcher.properties
• macOS and Linux: /etc/illumon.launcher.properties

Per-user properties file

The per-user properties file (.illumon.launcher.properties) is located in the user's home directory:

• Windows: %USERPROFILE%\.illumon.launcher.properties
• macOS and Linux: ~/.illumon.launcher.properties

Instance properties file

This is the .illumon.launcher.properties file located in an instance folder.

Properties relevant to the Deephaven Launcher