Configuration file locations

Deephaven processes obtain their configuration from the Configuration Server or from files stored on disk. "File" refers to actual files as well as chunks of configuration data that can be represented as a file.

Configuration in etcd

Much of a Deephaven system's configuration is stored in etcd and accessed via the Configuration Server:

Configuration file locations

When interacting with configuration files stored on the filesystem, there are several important points to keep in mind:

  1. The locations mentioned below are not part of a shared filesystem. This means that each machine in the deployment has its own copy of the files. To implement a configuration change, you must update the relevant file on all nodes, or on all nodes to which the change applies.

  2. Deephaven controls some files, which will be overwritten during an upgrade installation. Other files and directories are intended to persist through upgrades, and changes should only be made to those specific files.

  3. The installer sets up symbolic links between various files and directories to streamline upgrades and for historical reasons.

Most Deephaven configuration files can be found in the locations described below.

/etc/sysconfig/illumon.d is a link to the current customer configuration directory under /etc/sysconfig/deephaven. Inside this directory, many subdirectories are links into /etc/sysconfig/deephaven.

/etc/sysconfig/deephaven is managed by the installer. Most files are referenced through links under /etc/sysconfig/illumon.d, but there are a few exceptions.

The discussion below favors the "presented" locations rather than the "real" locations where there is an option.

/etc/sysconfig/illumon.d

This directory contains files specific to this installation. You can add or modify files in this directory, and your changes will be preserved during upgrades (except for hotfixes and overrides). This directory contains various files and subdirectories (though this is not an exhaustive list).

  • auth

    The auth directory has restricted access permissions. It contains files related to X509 certificates, keys, and passwords. The installer and utility scripts generally manage its contents.

  • auth-user

    The auth-user directory is similar to auth, but has filesystem permissions that allow all Deephaven processes read access.

  • calendars

    This is the default location for custom calendars.

  • chartthemes

    This is the default location for custom plotting themes.

  • client_update_service

    This directory contains files to customize the Client Update Service. This service prepares files for download by the Swing UI and other remote clients.

  • dh-config

    The files in dh-config configure connections to the configuration server(s). The files are managed by the installer and configuration packager.

  • etcd

    The files in etcd configure connections to the etcd service. The files are managed by the installer and configuration packager.

  • integrations

    Any files in integrations are synchronized to remote clients by the Client Update Service for optional integrations, such as R and numpy. The Integrations jar in /usr/illumon/latest/java_lib is also extracted and synchronized to clients. The integrations directory is not on the server classpath.

  • monit This directory is used only to preserve customizations which processes are automatically started by the m/monit process supervisor during a Deephaven upgrade.

  • plugins

    This directory contains files for Deephaven plugins.

  • resources

    This directory is on the classpath for Deephaven processes. You may modify existing files or add new files that can be loaded as resources.

    The truststore and password files for Deephaven processes are linked here from /etc/sysconfig/deephaven/trust. Truststore files do not contain secrets, and the password is used only for file validation.

    Warning

    Properties files are loaded from etcd, not from files in this directory.

  • java_lib

    JAR files found here take precedence over those in the Deephaven installation. See CLASSPATH for details.

  • override

    This path contains any class files that should override those contained in java_lib and hotfix JARs. See CLASSPATH for more details.

  • hotfixes

    This path contains any JAR files that should override those contained in all java_lib directories. See CLASSPATH for more details.

  • schema

    This is the initial location of Deephaven schemas loaded into etcd. The files are not used after installation. Schemas are stored in the configuration server. Use dhconfig schemas to manage them.

/etc/sysconfig/deephaven

Many files and directories in `/etc/sysconfig/illumon.d link to files under this directory.

  • monit

    The .conf files define the services running on this server. The Deephaven installer updates these files based on cluster.cnf, so any changes made to these files will be lost on upgrade. M/Monit configuration files link to files here from /etc/monit.d/deephaven.

/usr/illumon/latest

Default Deephaven system files are installed here. Default configuration files are in /usr/illumon/latest/etc. Do not modify these files.

Descriptions of selected files

hostconfig

The hostconfig file contains details that must be passed to processes as they are started, such as how much memory should be allocated, what account they should run under, and additional Java system properties they should use. All Deephaven services and other processes started with iris_exec or launch read this file.

The hostconfig file is /etc/sysconfig/illumon. This is where you should make any edits you expect to survive an upgrade. This editable file loads default settings from /etc/sysconfig/illumon.confs/hostconfig.system, which is written by the Deephaven installer and should not be modified.

If needed, additional processes to be managed by Monit can be added to the hostconfig file. If an existing process needs changes to allocated memory or other startup properties, those changes must be made in this file. See the Deephaven Operations Guide for additional documentation on Deephaven's use of Monit.

Tailer config file (tailerConfigDbInternal.xml)

The default tailer configuration file is /usr/illumon/latest/etc/tailerConfigDbInternal.xml. This is set by the property log.tailer.configs=tailerConfigDbInternal.xml and located on the CLASSPATH. See the Deephaven data tailer documentation for more information. To make changes to this configuration, copy the file to /etc/sysconfig/illumon.d/resources or change log.tailer.configs.

cluster.cnf

The cluster configuration file, /etc/sysconfig/deephaven/cluster.cnf, is written by the installer and should be edited only from the machine where the installation process is performed. Many Deephaven processes read configuration from this file when starting.

customer_jars.txt

JAR files and directories named in /etc/sysconfig/illumon.d/resources/customer_jars.txt are synchronized to remote clients.

authusers.txt

Deephaven users, groups and ACLs are stored in etcd or in a MySQL database. Plugins are available to authenticate with LDAP or SAML. In some situations, it is useful to maintain another set of Deephaven users in a local file. Enable this by setting the properties:

authentication.server.localusers.enabled=true
authentication.server.localusers.file=/etc/sysconfig/illumon.d/resources/authusers.txt

The authusers.txt file contains a list of Deephaven users in htpasswd format. Create new entries with htpasswd -nbm <username> <password>.

Note

This file is only relevant on machines running an authentication server.

superusers.txt

superusers.txt lists the local superusers and users allowed to operate as other users in the format <username>:<operate as user>. A * in the operate-as-user indicates that the username can operate as any user.

Enable this by setting the properties:

authentication.server.localsuperusers.enabled=true
authentication.server.localsuperusers.file=/etc/sysconfig/deephaven/superusers.txt

Note

This file is only relevant on machines running an authentication server.