Admin glossary

This page defines common terms used throughout the security, TLS, and other system administration docs, and explains how those terms relate to Deephaven installation and security.

For the current certificate, trust, routing, and certificate-rotation model, use TLS certificates as the primary operational reference.

DNS

The Domain Name System is what maps a hostname like localhost or www.example.com to an IP address.

On Deephaven servers, it is important that all members of the Deephaven cluster agree about what IP belongs to what hostname.

FQDN

A fully qualified domain name (FQDN) refers to a complete.hostname.address, and can be as simple as localhost.

In Deephaven, all Server processes in a cluster will communicate directly with other server processes using TLS connections to each machine's FQDN:<process port>.

For all Kubernetes installations and any Podman or native installation that opts in to using Envoy, the only FQDN and port your client machines will connect to is Envoy (default port 8000).

When Envoy is not used, a Deephaven client connects directly to each server's FQDN and configured port.

For how FQDNs affect certificate SAN requirements and routing, see Deephaven machine identity and the SAN and hostname verification section of the TLS certificates guide.

Machine identity

A Deephaven machine's machine identity is the FQDN other Deephaven services use when they connect to that machine.

In native installations, machine identity is configured in the cluster configuration and is used when Deephaven writes service endpoints, performs hostname verification, and validates or generates certificates for internal service-to-service communication.

All machines in a Deephaven cluster should resolve a given machine identity consistently. If different machines resolve the same Deephaven hostname to different IP addresses, internal TLS and token-validation flows can fail in ways that look like intermittent routing or authentication problems.

For the current native and Envoy-specific machine-identity rules, see Deephaven machine identity.

Deephaven client

A Deephaven client process is any Java, Python, JavaScript, C++ or R program that connects to a Deephaven server to access data, execute code or perform some administrative operation.

When Envoy is used, all Deephaven client connections to the server will pass through Envoy's FQDN:<envoy port> (default 8000).

Kubernetes installations always use Envoy, while Podman and native installations support opting in to using Envoy, but also allow direct connection from Deephaven clients to Deephaven services.

Deephaven services

Within a Deephaven cluster, there are one or more machines running one or more "Deephaven services" or "Deephaven processes".

Deephaven services all run as one of three Deephaven service accounts, by default irisadmin (admin processes), dbmerge (data writing processes) and dbquery (data reading processes).

Whenever a document refers to a "Deephaven service", it specifically refers to a Deephaven-authored Java process which is started and monitored by the Monit process manager. This includes all Administrative services, Data-accessing services, and Utility services.

Third-party Infrastructure services like Etcd and Envoy may run on Deephaven servers, but they are referred to by their own names rather than as "Deephaven services."

Deephaven service accounts

Deephaven services run as dedicated Operating System service accounts rather than as end-user accounts.

The three primary Deephaven service accounts are used in all installation types:

SettingDefault userPrivate groupPurpose
DH_ADMIN_USERirisadminDH_ADMIN_GROUP (irisadmin)Runs administrative services such as configuration and authentication services.
DH_MERGE_USERdbmergeDH_MERGE_GROUP (dbmerge)Runs merge processes that need write access to filesystem table data.
DH_QUERY_USERdbqueryDH_QUERY_GROUP (dbquery)Runs query processes and user workers with read access to table data and limited write access.

These accounts are collectively the main Deephaven service accounts and apply to Kubernetes, Podman, and native installations.

The following shared groups also apply across installation methods:

SettingDefault groupPurpose
DH_SHARED_GROUPdbmergegrpPrimary shared group for DH_ADMIN_USER, DH_MERGE_USER, and DH_QUERY_USER.
DH_SHARED_QUERY_GROUPdbquerygrpShared group for Deephaven query/session access beyond the core system accounts.

Note that all three of the primary Deephaven service accounts will have primary membership in DH_SHARED_GROUP, with secondary membership in DH_SHARED_QUERY_GROUP and their own "private group", defined above.

Some additional service accounts are specific to native installations:

SettingDefault user/groupPurpose
DH_MONIT_USER / DH_MONIT_GROUPirisadmin / irisadminRuns Monit, which starts and monitors Deephaven services. Requires permission to act as the three main service accounts.
DH_ETCD_USER / DH_ETCD_GROUPinstaller-definedRuns the Etcd service when Etcd is installed natively.
DH_SSH_USERinstaller-definedService account used by the native installer for SSH access during installation and maintenance.

For property definitions, defaults, and sudo/group requirements, see Custom user accounts.

Deephaven user

A Deephaven user is defined by user and group records stored and managed using Deephaven ACLs, and is not related to Operating System user accounts or Deephaven service accounts.

A Deephaven user logs in to Deephaven through the Authentication server, which is often accessed through the Web API service. Once the user has proven their identity, they are issued an authentication token which defines their logged in username, their operational username, and a secret token with a short expiration (default of 10 minutes).

Using their authentication token, the Deephaven user can make a wide variety of requests to the server, with each request authorized based on ACL permissions granted to the user, or any special groups the user is a member of.

When a Deephaven user requests to execute code, that code is executed on the host system by a Deephaven service account (typically dbquery, maybe dbmerge and never as irisadmin), with all authorization provided by the Deephaven user's authentication token.

Deephaven property

The primary means to configure a Deephaven cluster is through property files.

Property files use a robust format for defining scoped key-value properties. The files are stored in Etcd and are parsed and exposed to all Deephaven Services via the Configuration Server.

Whenever any document says to "set a property", unless otherwise specified, this means to edit a property file.

Administrative services

In every Deephaven installation, all authentication, authorization, system configuration, and Persistent Query execution is controlled by a set of "Administrative Processes":

Caution

Machines which perform administrative actions have administrative secrets in secure locations on disk. You should isolate Query and Merge dispatchers from machines which run any administrative process. Even if your secrets on disk are safe, each machine receives a TLS certificate which can sign for that machine's qualified domain name, so you should not allow any untrusted users to execute code on these machines.

Data-accessing services

The finer nuances of controlling read and write access to user data are better covered at a high level in the Data lifecycle guide, and in verbose detail in the ACL guide.

With respect to security, we only need to consider three specific "data-accessing processes":

  • Query dispatchers (dbquery process)
    • Read-intensive Persistent Queries and console workers which can execute user Groovy or Python code.
    • Allows the least-trusted users access to historical and live data sources.
    • Run query dispatchers in isolated environments with read-only access to filesystem data.
    • You may choose to give some or all query users access to writable user tables using Deephaven group ACLs.
  • Merge dispatchers (dbmerge process)
    • Merge workers run with filesystem permission to write to shared data directories.
    • Starts Persistent Queries and console workers that can execute user Groovy or Python.
    • May import very many rows of user data in a short period of time (typically via CSV or JDBC).
    • May steadily write very many rows of data over a long period of time.
    • Restrict Deephaven user access to trusted users who are members of the iris-data* ACL "special groups".
  • Data Import Servers (db_dis process)
    • Listens for data sent by Log tailer processes to write data in "near realtime" to "Intraday" or "live" data locations (typically /db/Intraday).
    • Writes very many rows of data from multiple sources over a long period of time.
    • When using multiple Data Import Services or merge dispatchers to distribute load, it is important to share network filesystems securely.

For secure, high-performance systems you may want to run one or more of each data-accessing process on a different machine, while more economical clusters often run db_dis and dbmerge on the same server. For simple test machines, all processes can run on a single machine, with Deephaven ACLs and OS file system permissions used to prevent unauthorized access to data.

Utility services

Some Deephaven processes exist solely to support collecting and distributing data in the system.

All installation methods, by default, run a Log Tailer process on all machines and in all containers to ensure that all binary log files written to disk are forwarded to the Data Import Server.

For Kubernetes, all processes are managed automatically, and run in their own pod(s). The log tailer runs in a sidecar container.

For native installations, use cluster.cnf to assign processes to specific machines. Native installations automatically add a Log Tailer, a Log Aggregator Service, and a Table Data Cache Proxy by default, but this can be manually disabled via node roles in cluster.cnf. The only exception to this rule is for Etcd-only or Envoy-only nodes; if no other Administrative or Data-accessing services are running, no log tailer or log aggregator will be added by default.

Infrastructure services

Deephaven relies on some third-party binary processes which may or may not run on your Deephaven servers.

  • Etcd
    • Stores all global cluster configuration data, ACLs and runtime system state.
    • Must be accessible to all other Deephaven services.
    • Need not be accessible to any Deephaven client processes.
    • For optimal performance and security, run Etcd on its own machine (with a fast-writing disk) and no other processes.
  • Envoy
    • Simplifies firewalls by exposing a single hostname and port to all client processes.
    • For optimal performance, co-locate Envoy with the Administrative services.
  • MySQL
    • The legacy datastore used only for table and data ACLs.
    • Superseded by Etcd-based ACLs since 2023.
    • No longer supported after the Grizzly+ release.

For Kubernetes, all services run in their own pod. Envoy and etcd are always used, MySQL is never used. For Podman, Etcd and, optionally, Envoy run within the single dh-infra pod. MySQL is never used. For native installations, the placement of every process is configurable via cluster.cnf. MySQL and Envoy are unused by default.

Deephaven installation type

Deephaven can be installed on Kubernetes, Podman or natively installed on a Linux host.

Choosing an appropriate installation method depends on many factors that do not belong in a glossary. Contact a Deephaven representative to plan an installation method that works for you.

TLS concepts

These entries define the TLS terms used throughout the docs. For Deephaven-specific certificate requirements, trust boundaries, routing models, and certificate rotation workflows, see TLS certificates.

Encryption vs authentication

Encryption protects data in transit from eavesdropping.

Authentication can happen at different layers. In TLS, it usually means the client verifies the identity of the server. At the application layer, it means a user or client proves its identity to obtain a session or token.

In a typical HTTPS/TLS connection:

  • The server presents a certificate to prove its identity.
  • The client verifies that certificate against its trusted CAs and checks that the hostname matches.
  • Once the TLS handshake completes, the connection is encrypted using negotiated session keys.
  • Credentials, tokens, and other sensitive application data can then be sent over that encrypted connection.

Deephaven provides multiple TLS boundaries between servers and clients, and encrypts sensitive data in transit.

Public and private keys

  • A private key is secret material that must be protected. It is used to prove identity (and to decrypt/sign, depending on the key type and negotiated TLS parameters).
  • A public key is shared. It is embedded in a certificate.

A server’s certificate is only useful when paired with its corresponding private key.

Certificate (X.509)

An X.509 certificate is a signed document that binds:

  • an identity (the subject), and
  • a public key,

plus metadata and constraints (validity period, key usage, etc.).

Certificates are usually used in TLS for:

  • server authentication (the browser/application confirms it is talking to the intended server), and optionally
  • client authentication (mutual TLS).

Certificate Authority (CA)

A Certificate Authority (CA) issues (signs) certificates.

  • A root CA is typically self-signed and is the anchor for trust.
  • An intermediate CA is issued by a root (or another intermediate) and issues leaf/server/client certificates.

Certificate chain (leaf + intermediates)

A certificate chain is the set of certificates needed for a verifier to build a path from the presented leaf certificate back to a trusted root CA.

Operationally:

  • Servers commonly present the leaf certificate plus any required intermediate certificates.
  • Clients validate the chain and then decide whether the root is trusted (via a truststore).

SAN

  • SAN (Subject Alternative Name) is the certificate extension that carries DNS names (and/or IP addresses) the certificate is valid for.
  • CN (Common Name) is an older subject field which predates the SAN extension.

In Deephaven, all hostname verification uses SAN. Common Name is no longer used.

Practically speaking, this means all TLS certificates for a given machine must contain a DNS:your.fqdn.here entry for the hostname used to reach that machine. See SAN and hostname verification for the current Deephaven rules.

Hostname verification

Hostname verification is the check that the hostname you connected to (for example, deephaven.example.com) matches a name in the certificate’s SAN extension.

Typical failure symptoms include hostname-mismatch errors indicating that the FQDN the client used is not listed in the certificate's Subject Alternative Name. Trust-chain failures such as PKIX path building failed are related TLS problems, but they indicate a broader trust or certificate-chain issue rather than the hostname-verification check itself. For Deephaven-specific behavior, see SAN and hostname verification.

External TLS trust

External TLS trust is the trust relationship between a Deephaven client and the Deephaven endpoint it reaches.

When Envoy fronts Deephaven, the client validates the Envoy endpoint certificate. When Envoy is not used, the client validates the certificate presented directly by the Deephaven service it connects to.

See TLS trust boundaries and request routing with Envoy for the current Deephaven model.

Internal TLS trust

Internal TLS trust is the trust relationship Deephaven services use when they connect to each other.

Kubernetes has long had a clear internal-versus-external TLS split. In Grizzly+, native installations formalize the same model more clearly through per-machine certificates, stricter internal trust, and optional generated internal certificates.

See TLS trust boundaries and internal TLS trust for the current Deephaven model.

External truststore

An external truststore is the truststore used for client-facing TLS trust.

In native installations that generate client truststores from explicit CA material, this is the familiar truststore-iris.p12. In Deephaven properties, tls.truststore refers to the client or default truststore used for external trust.

See external TLS trust and optional truststores for the current Deephaven behavior.

Internal truststore

An internal truststore is the truststore Deephaven services use for internal service-to-service TLS trust.

In native installations that generate internal certificates, this truststore is separate from the external truststore and is referenced by the internal.tls.truststore property. This allows internal trust to be narrower than client-facing trust.

See internal TLS trust and optional truststores for the current Deephaven behavior.

Trust and truststores

Trust is the decision by the client/application that a certificate chain is acceptable.

A client generally trusts a server certificate if:

  1. the certificate chain is valid (not expired; signatures verify; required intermediates are present),
  2. the certificate is allowed for its intended usage (key usage / extended key usage),
  3. hostname verification succeeds (SAN matches), and
  4. the chain terminates at a root CA that the client trusts.

A truststore is the database of trusted root CA certificates (and sometimes intermediate certificates).

Truststore

A truststore is a set of certificates (typically CA roots, sometimes intermediates) used to decide whether a presented certificate chain is trusted.

In these docs, “truststore” may refer to either a Deephaven-managed truststore generated from explicit CA material or a system-managed OS/browser/JVM/Python trust source.

Examples include:

  • a Deephaven-managed truststore generated during installation or upgrade when you provide a CA bundle of trusted PEM certificates;
  • an OS/browser/JVM truststore (system-managed)
    • For OS trust, Linux Operating Systems have a directory of individual certificates, and a single bundled pem certificate.
      • To find your OS truststore, OpenSSL provides a reliable symlink: ls -l /etc/ssl/cert.pem
      • Your system administrator can update Ubuntu truststores via update-ca-certificates and RedHat OSes via update-ca-trust
    • For JVM trust, your vendor may use OS trust, or a cacerts file installed with the JVM.
    • For Python trust, most libraries embed their own *.pem in their site-packages.
    • For browsers, consult your system administrator before making any changes.

See also:

System CA

System CA means trust comes from the default OS/browser/JVM/Python truststores on the machines involved, such as the system’s installed root CA certificates.

Practically:

  • You rely on what those environments already trust.
  • If your certificates are issued by a private/corporate CA, that CA must be installed into the relevant system/JVM truststores on clients and/or servers.
  • Certificate rotation is handled by your system administrator.

See also:

Customer CA

Customer CA means you provide an explicit CA bundle to act as the trust anchor, for example via DH_CA_CERTIFICATE in the native installer. Certificates used in that trust model must chain back to, and be trusted by, the provided CA bundle.

Practically:

  • This is a common approach when certificates are issued by an internal PKI.
  • The CA bundle you provide becomes the trust anchor used to generate Deephaven-managed truststores.
  • Certificate rotation is handled by updating CA certificate files on all servers, restarting all processes, then updating all clients.

See also:

Generated CA

Generated CA means Deephaven-generated internal trust material used for internal service-to-service TLS.

Practically:

  • Internal trust is restricted to the generated trust anchor rather than the system truststore.
  • External client-to-cluster TLS still uses the Envoy or service certificate you provide.
  • In the native installer, this corresponds to generated per-machine certificates and DH_GENERATE_CERTIFICATES=true, which is supported only with Envoy.
  • Both the generated certificates and the internal trust material are managed by Deephaven and updated during upgrade or certificate rotation workflows.

See also:

System trust vs a custom truststore

Clients may rely on:

  • System trust: OS, browser, JVM, or Python defaults.
  • A custom truststore: an explicit set of trusted CA certificates.

If Deephaven is configured to use certificates issued by an internal corporate CA, that CA typically must be added to client system truststores or distributed through an explicit truststore.

This doc set also references Deephaven-managed truststores:

  • Deephaven supports one external client trust model at a time:
    • System trust, or
    • Deephaven-managed truststores generated from an explicit CA bundle provided during Deephaven installation and upgrades.
  • When Deephaven-managed truststores are generated, the truststore files are replaced on disk, rather than merged with older trust material.
  • The truststore passphrase file name referenced in these docs is truststore_passphrase.

See also: Optional truststores.

Keystore (private keys + certificates)

A keystore is a file/container format used by some software stacks to store:

  • private keys, and
  • certificates (often including the chain).

It differs from a truststore, which stores certificates used for trust decisions (usually CA roots).

File/container formats: PEM, DER, PKCS#12

  • PEM: base64-encoded text with -----BEGIN ...----- markers. Common for certificates and private keys.
  • DER: binary encoding of the same ASN.1 structures.
  • PKCS#12 (.p12/.pfx): a binary container commonly used for keystores/truststores.

All certificates supplied to a Deephaven installation are supplied in PEM format. For Deephaven-specific certificate content requirements, see What a Deephaven server certificate must contain.

To convert a DER certificate to PEM:

Private key login "keypair files"

Deephaven non-interactive private key login uses two related "public key" and "private key" files, to perform "challenge-response" authentication by signing a nonce with the private key and verifying it with the public key.

  • Public key file (pub-*.base64.txt): contains only the Deephaven username and the public key. This file is safe to distribute and is used to authorize the user on the server.
  • Private keypair file (priv-*.base64.txt): contains the Deephaven username, optional operateas, and both the public and private key material. This file is a credential and must be protected with restrictive filesystem permissions.

See Private key login key files (including Rotation and revocation) for file formats and operational guidance.

A Java or Python client uses a private keypair file by supplying its path via the Java system property AuthenticationClientManager.defaultPrivateKeyFile.