File system backup and restoration

Though a majority of configuration is stored in etcd, there are several files and directories containing additional configuration stored under /etc/sysconfig/deephaven. File-on-disk configuration tends to be related to process startup, authentication, and additional JARs/classes on the classpath to extend Deephaven's functionality.

Process startup configuration

Process startup override parameters such as heap are stored in etc/sysconfig/deephaven/illumon.iris.hostconfig. This file exists on each node within a cluster and may have different values for a given process between nodes. For example, the tdcp process on one node within the cluster may be assigned 6G of heap, whereas the tdcp process on a different node may be assigned 10G of heap.

If a custom tailer exists, configuration XML should be stored in /etc/sysconfig/illumon.d/resources, though this may vary between deployments.

Authentication files

Many authentication keys and certificates are stored under /etc/sysconfig/deephaven/auth and /etc/sysconfig/deephaven/auth-user.

/etc/sysconfig/deephaven/auth is used to store sensitive certificates and keys, and should not be accessible to most users. Particular files of interest include:

  • priv-*.base64.txt: These files contain system users' private keys, which allow key-based access to the system. If required, new keys may be generated for users.
  • /etc/sysconfig/deephaven/auth/dsakeys.txt: If you have enabled text-file based key authentication (not enabled by default), dsakeys.txt may be used to define key-based authentication for users. This file may be empty or missing if public keys are stored in etcd.
  • /etc/sysconfig/deephaven/auth/authusers.txt: This file will exist if you have enabled text-file password-based authentication (not enabled by default).
  • Any custom certificate files needed for authentication with external services such as LDAP are stored in /etc/sysconfig/deephaven/illumon.d.latest/resources/.

Note

It is a good safety measure to back up all of /etc/sysconfig/deephaven/auth to ensure that needed keys and passphrase files are available in case they are later needed for restore operations.

The /etc/sysconfig/deephaven/auth-user directory is used to store the webServices-keystore.p12 certificate used by the web service, and is generally visible to all Deephaven query and merge processes. Copying this file to a different cluster is not recommended because it may be hostname-specific. Instead, a new certificate should be used (or generated) during the installation of the new cluster.

Plugins and custom jars

Deephaven is highly customizable, and installations may have customized JARs/classes and plugins that extend the functionality of the system. Generally speaking, customized JARs and classes will be in /etc/sysconfig/deephaven/illumon.d.latest/java_lib/.

Plugins should not be copied between Deephaven instances; instead, they should be re-installed and re-activated in the new environment. Activated plugins can be seen in /etc/sysconfig/deephaven/illumon.d.latest/plugins/.