Multi-node configuration
In multi-node Deephaven deployments, configuration files such as certificates, truststores, etcd credentials, and connection settings must be synchronized across all servers. The config_packager.sh utility creates and extracts tar archives containing these files, simplifying the distribution process.
Location: /usr/illumon/latest/install/config_packager.sh
When to use config_packager.sh
Use config_packager.sh when you need to:
- Set up additional nodes in a cluster after initial installation.
- Replace or update TLS certificates across all nodes.
- Recover from etcd cluster failures.
- Add new etcd nodes to an existing cluster.
Note
During initial cluster installation and regular upgrades (such as certificate rotation), the Deephaven installer handles configuration distribution automatically. Manual use of config_packager.sh is primarily for manual maintenance and disaster recovery scenarios.
Package types
The utility supports four package types, each containing a different set of configuration files:
| Package | Archive name | Contents |
|---|---|---|
| trust | iris_trust.tgz | Truststores, truststore passphrase, Configuration Server connection files |
| query | dh_query.tgz | Everything in trust, plus TDCP login private key, controller encryption keys, and etcd client credentials |
| auth | iris_auth.tgz | Everything in query, plus additional etcd credentials |
| etcd | dh_etcd_config.tgz | etcd server and client configuration files (created by config_generator.sh, unpacked by config_packager.sh) |
Choosing the right package type:
- Use trust packages for nodes that only need to verify certificates and connect to the Configuration Server (minimal footprint).
- Use query packages for query server and merge server nodes that need TDCP login keys, controller encryption keys, and etcd client access.
- Use auth packages for nodes running administrative services that need additional etcd client credentials.
- Use etcd packages for etcd cluster configuration, including server setup and client credential distribution.
Prerequisites
Run the script as the installation service account, which has the correct permissions for all operations:
Basic workflow
The typical workflow for distributing configuration is:
- Package configuration on the source machine (usually the infrastructure server).
- Copy the archive to target machines using
scpor similar. - Unpackage on each target machine.
Usage
Syntax:
Types:
trust— Package or unpackage trust configuration (truststores, Configuration Server connection files).query— Package or unpackage query server configuration (everything in trust, plus TDCP keys and etcd client credentials).auth— Package or unpackage authentication configuration (everything in query, plus additional etcd credentials).etcd— Unpackage etcd server or client configuration (packaging is handled byconfig_generator.sh).
Actions for auth, trust, and query packages:
| Action | Description |
|---|---|
package | Create a tar archive in the current directory. |
unpackage | Extract and apply files from a tar archive in the current directory. |
Optional mode for auth, trust, and query packages:
An optional third argument specifies the packaging mode:
lite(default): Skips private keystores (.p12) and their passphrases. This is the standard mode used by the installer.full: Includes all files, including private keystores and passphrases. Use only for disaster recovery scenarios where keys cannot be regenerated on the target machine.
Actions for etcd packages:
| Action | Description |
|---|---|
unpackage-server <n> | Unpackage etcd server configuration for server number n. |
unpackage-client | Unpackage etcd client configuration. |
unpackage-global-client | Unpackage etcd client configuration for all standard client roles. |
The unpackage-global-client action installs etcd credentials for: controller, dispatcher, dispatcher-ro, and dh-resolver. These are the roles needed by query server nodes.
Note
The etcd package is created by config_generator.sh, not by config_packager.sh. The config_packager.sh utility only handles unpacking etcd configuration.
Examples
Distribute query configuration to a new node
On the source machine (infrastructure server):
Copy to the target machine:
On the target machine:
Distribute etcd configuration to cluster nodes
After generating etcd configuration with config_generator.sh, distribute to each etcd server:
The server number corresponds to the order in which IP addresses were specified when running config_generator.sh (1-indexed).
Update trust configuration after certificate changes
Backup behavior
Before extracting files, config_packager.sh automatically backs up existing configuration to /etc/sysconfig/deephaven/backups/. Each backup is timestamped (for example, trust_package.2025-06-03T110530.12345/), allowing you to recover previous configurations if needed.
To restore from a backup, manually copy the files from the backup directory back to their original locations (such as /etc/sysconfig/deephaven/auth/ or /etc/sysconfig/deephaven/trust/).
Logging
The script logs all operations to /var/log/deephaven/install_configuration/. The most recent log for each operation type is symlinked as current.config_packager_<type>_<action>.log (for example, current.config_packager_query_package.log).