Deephaven Launcher and Client configuration

Client Components Update

When a new Deephaven console is launched from the Launcher, the Launcher will check whether the instance has the latest versions of JAR files and other files from the Client Update Service (CUS) associated with the instance. It will download newer files if updates are available. Once the console is running, it will check for changes every 10 minutes. If new files are found (because an update has been applied or a configuration file changed on the server), a message banner will appear on the client console to inform the user that the console is out of date. A relaunch of the console will then be required to download and apply the new files. Continuing to run the console with out-of-date components may appear to work fine. However, problems may arise when client-side settings are not correct for the current environment, or when the client is using classes that no longer match the versions installed on the server.

The directory /var/www/lighttpd/iris/iris/ contains the files used to control the Client Update Service. This directory is populated by syncing files from system directories as well as etcd, and is rebuilt every time the client_update_service process is restarted; these files should never be edited. If needed, previous versions of the /var/www/lighttpd/iris directories can be found in /var/www/lighttpd/backups; these backups are created every time Deephaven is upgraded, and can be safely discarded once the new version has passed testing.

The getdown.txt file on the server (/var/www/lighttpd/iris/iris/getdown.txt) controls most of the Client Update Service behavior. The getdown.txt file on the server (/var/www/lighttpd/iris/iris/getdown.txt) controls most of the Client Update Service behavior. This file is dynamically built by the Client Update Service when it starts, using files from the directory /etc/sysconfig/illumon.d/client_update_service and adding required resources from the Deephaven installation. This directory is also recreated each time the Deephaven software is installed, but existing files are copied to the new directory.

/var/www/lighttpd/iris/iris/getdown.txt is constructed by concatenating two files:

• a system-owned /var/www/lighttpd/iris/iris/getdown.txt.pre, which is automatically replaced every time Deephaven is upgraded.
• a customer-owned /etc/sysconfig/illumon.d/client_update_service/getdown.global, which is never replaced, and where all customer-controlled configuration should be placed.

For a comprehensive explanation of system vs. customer files, see the Client Update Service documentation.

The customer-owned files are as follows:

• getdown.host - this contains the host address, including an optional port.
• getdown.protocol - this contains the expected protocol, typically https.
• getdown.global - this contains all the parameters used by getdown and contains most of the parameters copied to getdown.txt.
• getdown.java.1.8, getdown.java.11, getdown.java.13, getdown.java.17 - these files contain default JVM parameters that will be use by clients based on the Java installation used.
• getdown.txt contains the settings detailed below (copied from getdown.txt.pre and getdown.global), and also a list of files to be downloaded. The contents of the generated getdown.txt file and all files to be downloaded are hashed and the values saved in digest2.txt. This is the file clients use to see whether there are updates to be applied, and to know what files to download.

When any of the files needed by the client are updated, you must restart the Client Update Service to regenerate the configuration files and digest. CUS is restarted automatically after a Deephaven product RPM is applied to a query server. The updated digest2.txt will allow running clients to see that updates are available when they next check for changes.

The script /usr/illumon/latest/cus/bin/cus is responsible for concatenating the system-owned /var/www/lighttpd/iris/iris/getdown.txt.pre with the customer-owned /etc/sysconfig/illumon.d/client_update_service/getdown.global to form the final configuration file, /var/www/lighttpd/iris/iris/getdown.txt, which is what getdown uses to decide what to send to client machines. The CUS script is also responsible for copying the client binaries and resources from the latest Deephaven release, the Deephaven customer and plugin directories as well as prop files from etcd into the web directory to be distributed to client machines.

Customer-Updateable files

The main settings and sections in getdown.global and their purposes follow.

appbase

This indicates the URL from which all subsequent files will be downloaded. All other files are relative to this root. This is generated from getdown.protocol, getdown.port, and getdown.host files, and should never be directly edited.

appbase = https://irisprod.company.com/iris

To change the appbase, you should instead edit getdown.protocol, getdown.host, and getdown.port, as detailed in the Client Update Service documentation.

The default page (index.html) for the parent of this path contains overview information about the Deephaven client and provides installer downloads. This page can be customized with environment-specific information, such as whom to contact for access to Deephaven.

ui.install_error

Certain errors will point to this page for more information. The default page provides contact information for Deephaven Data Labs support. It can be customized to add internal support information, if desired. Again, this is generated from getdown.protocol and getdown.host, and should never be directly edited. To change this value, follow the instructions in the Client Update Service documentation.

ui.install_error = https://irisprod.company.com/iris/error.html

ui.name

The default value is ui.name = Deephaven.

However, users can customize this setting. For example:

ui.name = Deephaven Data Labs for Company Name Here

This will be the title of the application while downloading files.

Configuration.rootFile

This sets the properties file to be used by the client. The default is iris-common.prop.

jvmarg = -DConfiguration.rootFile=iris-common.prop

server_java_version

The Deephaven Launcher reads the property server_java_version = xx from getdown.txt, and will require that Deephaven be started with a matching Java version, where matching means that the version from java -version starts with xx. The required version comes from /usr/illumon/latest/etc/JAVA_VERSION, and the property values come from /etc/sysconfig/illumon.d/client_update_service/getdown.java.xx. This functionality requires Launcher 8.14 or higher, and appropriate settings on the server. The property can be added manually to the getdown.java.xx file on the server to enable this feature.

The Launcher will try the version used to run the Launcher first. If that doesn’t match, it will present dialogs allowing the user to select a new JDK. One option is to continue with an "invalid" version, primarily to handle future unforeseen situations.

Non-Updateable files

The main settings and sections in getdown.txt.pre and their purposes follow. These cannot be edited.

classpath

Resources will be searched for in the order these directories are listed, including searching inside JARs. Most Deephaven files are in the resources and java_lib directories. There are separate settings for classpaths on Windows, Linux, and OSX.

jvmarg = -classpath

jvmarg = [windows] "%APPDIR%\private_classes;%APPDIR%\private_jars\*;%APPDIR%\override;%APPDIR%\hotfixes\*;%APPDIR%\config;%APPDIR%\resources;%APPDIR%\java_lib\*;%APPDIR%\plugins\hotfixes\*;%APPDIR%\plugins\override;%APPDIR%\plugins;%APPDIR%\plugins\*;%APPDIR%\customer_jars\*"

jvmarg = [linux] %APPDIR%/private_classes:%APPDIR%/private_jars/*:%APPDIR%/override:%APPDIR%/hotfixes/*:%APPDIR%/config:%APPDIR%/resources:%APPDIR%/java_lib/*:%APPDIR%/plugins/hotfixes/*:%APPDIR%/plugins/override:%APPDIR%/plugins:%APPDIR%/plugins/*:%APPDIR%/customer_jars/*

jvmarg = [mac] %APPDIR%/private_classes:%APPDIR%/private_jars/*:%APPDIR%/override:%APPDIR%/hotfixes/*:%APPDIR%/config:%APPDIR%/resources:%APPDIR%/java_lib/*:%APPDIR%/plugins/hotfixes/*:%APPDIR%/plugins/override:%APPDIR%/plugins:%APPDIR%/plugins/*:%APPDIR%/customer_jars/*

%APPDIR%/config/*:%APPDIR%/private_classes:%APPDIR%/private_jars/*:%APPDIR%/override/*:%APPDIR%/resources:%APPDIR%/java_lib/*

clean_target and clean_target_resource

These specify directories under the instance directory whose contents should be completely replaced when updating. This is required for parts of the classpath that use wildcards to find JARs whose names will change from version to version, and to handle resources that are removed. Any files in directories named by a clean_target or clean_target_resource line will be removed unless they are named on a code = or resource = line. Note: Deephaven Launchers with version <= 8.15 do not look at resource = when handling clean_target, and do not process clean_target_resource. Directories that might contain resource = resources should use clean_target_resource.

clean_target = java_lib
<numerous clean_target entries follow>

clean_target_resource = calendars
clean_target_resource = chartThemes

While it is possible to add additional values for clean_target and clean_target_resource to getdown.global, in practice, it is very rare to need to do so. Do NOT edit targets in getdown.txt.pre, as they will be lost upon upgrading.

Code and resources

Code and resource entries in getdown.txt identify specific files for which digest values should be calculated, and that the Launcher should pull down to a client. The base path for these files is /var/www/lighttpd/iris/iris. The script /usr/illumon/latest/cus/bin/cus rsyncs several server directories there prior to regenerating getdown.txt and updating the digest. The default paths are:

• /usr/illumon/latest/java_lib//etc/sysconfig/illumon.d/java_lib/
• /etc/sysconfig/illumon.d/override/
• /etc/sysconfig/illumon.d/hotfixes/
• /etc/sysconfig/illumon.d/integrations/
• /etc/sysconfig/illumon.d/resources/
• /etc/sysconfig/illumon.d/calendars/
• /etc/sysconfig/illumon.d/chartthemes/
• /etc/sysconfig/illumon.d/dh-config/

The script then goes on to exclude some items, such as configuration files not needed by client installations. It will also include all the prop files stored into etcd; if you update your props in etcd, you will need to restart the client_update_service process to have those new files sent to client machines.illumon.d.

Additional files that are needed by Launcher clients should usually be added to /etc/sysconfig/illumon.d/java_lib/ or /etc/sysconfig/illumon.d/resources/. However, this will add the files to the classpath of server processes as well. Alternatively, custom individual code entries could be added to getdown.global.

Like many other text configuration files, getdown.global, getdown.txt, and getdown.txt.pre use the pound/hash symbol [#] to indicate comment lines.

Uploading custom scripts and resources to Swing Clients

It may be required to execute a custom script on the swing clients, for example to download jars from a repository. Other resources can also be uploaded to clients. This can be done as follows.

The directory /etc/sysconfig/illumon.d/client_update_service/resources contains additional files which will be copied to the getdown directory /var/www/lighttpd/iris/iris. All this should be done as the Deephaven administrative user, typically irisadmin.

1. Create your additional files in the directory. For example, to create a script to be executed on the swing clients two files will be needed. The first is the script, which in this example is called prelaunch.sh:

   #!/usr/bin/env bash
   <Script commands>
   

2. The second must be called prelaunch.txt, and contains commands to be executed by the uploading client. A shell command is specified with exec. getdown won't create the script with executable permissions, so this must be done as a command. In this example:

   exec = chmod 754 ./prelaunch.sh
   exec = ./prelaunch.sh
   

3. Update /etc/sysconfig/illumon.d/client_update_service/getdown.global to include the new resources.

   # prelaunch file
   resource = prelaunch.txt
   resource = prelaunch.sh
   

4. Restart the CUS:

   sudo monit restart client_update_service
   

Launcher properties files

Properties files and JAR files are downloaded automatically from the Deephaven server by the Launcher, so no persistent changes can be made by editing these files on the client. To make global changes, make the changes on the server and restart the client_update_service.

To apply configuration changes to the properties of specific client installations, these can be made by adding property keys and values to one of the jvmargs.txt files. There are several copies of this file that have varying levels of applicability depending on the location:

• <WorkspaceRoot>/jvmargs.txt - applies to all instances
• <WorkspaceRoot>/<instance_name>/jvmargs.txt - applies to all workspaces and consoles in the instance
• <WorkspaceRoot>/<instance_name>/workspaces/<workspace_name>/jvmargs.txt - applies to this Deephaven workspace
• <WorkspaceRoot>/<instance_name>/workspaces/console/jvmargs.txt - applies to consoles in the instance

Settings from all applicable files will be combined. If there are settings that exist in more than one of the files, the last one evaluated will take precedence. They are evaluated in the order listed above. The settings in the jvmargs.txt files also override settings from properties files.

The format of jvmargs.txt is similar to other properties files in that blank lines are ignored and comments start with a pound/hash [#] symbol. Because this file is being used to pass extra arguments into the Java command line, most of its key=value entries will start with -D. For example:

-DCsvExport.useFileSelectionDialog=true

Client Update Service script

Timeout

In some instances, the CUS may take longer than 30 seconds to start. By default, monit ends processes after 30 seconds. To avoid this, the default timeout is 90 seconds:

check process client_update_service with pidfile /var/run/illumon/cus.pid
    start program = "/etc/init.d/cus start /path/to/logs" with timeout 90 seconds
    stop program = "/etc/init.d/cus stop /path/to/logs"

Note that the timeout is pure monit functionality and actually applies to all services. If needed, customers can modify the "with timeout xx seconds" argument to increase the time monit gives the process.

Logs

The CUS script creates persistent log files. The default log location is /var/log/deephaven/misc. If a different location is desired, the path can be added to the start and stop program lines as above. These logs can be used to diagnose issues or be sent to Deephaven support.

Installation Location Information

Three types of files are stored during Deephaven Launcher installation and operation:

• Deephaven Launcher program files - These are the files required to run the Deephaven Launcher. These files do not change, and they do not contain data, downloaded JARs, or workspace files. On Windows clients, these files are installed on a per-user basis or per-machine basis, depending on the installation option chosen.
• Program files (also known as instance files) - These files include the JARs and the configuration files downloaded and synchronized for each instance, as well as log files generated by the Deephaven Launcher and Console programs.
• Workspace files - These files are workspace settings, backups, and history files.

The Deephaven Launcher displays the locations of Workspaces, Program, and Log files at the bottom right, as shown here:

Deephaven Launcher program files

Whether or not the Deephaven Installer is used for the installation, the program will be started from a working directory containing the shell script or executable, and supporting JAR and resource files.

Program files

A Deephaven instance specifies the server and port to which Deephaven should connect for that session. Users can configure multiple instances in the Deephaven Launcher.

The <programfiles>/<instance>/getdown.txt file enables an instance to appear as a choice in the Launcher. For example, if programfiles/foo/getdown.txt exists, then "foo" will be listed as an instance.

By default, program files and log files are stored under the following locations:

• Windows: %LOCALAPPDATA%/illumon
• macOS and Linux: ~/iris/.programfiles

Workspace files

Multiple workspaces can be created for each Deephaven instance. In the Launcher, the Workspace drop-down menu will list all available Workspaces for the selected Instance.

By default, workspace files are stored under the following locations:

• Windows: ~/Documents/Iris
• macOS and Linux: ~/iris/workspaces

For example, instance "foo" and workspace "Default" will be in ~/iris/workspaces/foo/workspaces/Default.

Deephaven Launcher Properties files

When the Launcher starts, it reads properties from all the launcher properties files it can find. Property values will be updated to replace a leading ~ or ${user.home} with the user's home directory. Files will be loaded in the order shown below. Later property settings override earlier ones, and system properties always have precedence.

Deephaven Launcher program properties file

The Launcher properties file (illumon.launcher.properties) is located in the Deephaven Launcher program folder.

System-wide properties file

The system-wide properties file (illumon.launcher.properties) is located in the Deephaven Launcher program folder:

• Windows: %PROGRAMDATA%/illumon/illumon.launcher.properties
• macOS and Linux: /etc/illumon.launcher.properties

Per-user properties file

The per-user properties file (.illumon.launcher.properties) is located in the user's home directory:

• Windows: %USERPROFILE%\.illumon.launcher.properties
• macOS and Linux: ~/.illumon.launcher.properties

Instance properties file

This is the .illumon.launcher.properties file located in an instance folder.

Properties relevant to the Deephaven Launcher