Deephaven public and private keys

Authentication server

The Authentication Server’s keystore contains the certificate and private key of a TLS enabled service. You must keep this file private, and not distribute it to clients. Each node running an Authentication Server should have its own keys, with a certificate that is signed (issued) by the CA in the TLS truststore file (see below).

The default self-signed key pair for the Authentication Server is generated and saved to a PKCS12 (.p12) keystore file when iris_keygen.sh is run. The keystore file is owned by user irisadmin with permission set to 400:

[-r-------- irisadmin irisadmin]  keystore.authserver.p12

The keystore file is protected by a unique randomly generated password stored in base64 encoded format in a read-only hidden file owned by user irisadmin with permission set to 400:

[-r-------- irisadmin irisadmin]  .auth_passphrase

Properties file sections

These file locations are specified by the following properties, which need to be accessible in the authentication_server and db_acl_write_server contexts.

[service.name=db_acl_write_server|authentication_server] {
    tls.keystore=/etc/sysconfig/illumon.d/auth/keystore.authserver.p12
    tls.passphrase.file=/db/TempFiles/irisadmin/.auth_passphrase
}

Configuration server

The Configuration Server’s keystore contains the certificate and private key of a TLS enabled service. You must keep this file private, and not distribute it to clients. Each node running a Configuration Server should have its own keys, with a certificate that is signed (issued) by the CA in the TLS truststore file (see below).

The default self-signed key pair for the Configuration Server is generated and saved to the .p12 keystore file when iris_keygen.sh is run. The keystore file is owned by user irisadmin with permission set to 400:

[-r-------- irisadmin irisadmin] keystore.configuration_server.p12

The Configuration Server keystore file is also protected by a unique randomly generated password stored in base64 encoded format in a read-only hidden file owned by user irisadmin with permission set to 400:

[-r-------- irisadmin irisadmin] .configuration_server_passphrase

Properties file sections

These file locations are specified by the following properties, which need to be accessible in the configuration_server context:

[service.name=configuration_server] {
    configuration.server.ssl=true
    configuration.server.keyfile=/etc/sysconfig/illumon.d/auth/keystore.configuration_server.p12
    configuration.server.passphrase.file=/db/TempFiles/irisadmin/.configuration_server_passphrase
}

Web API Service keystore

The Web API Server's keystore contains the certificate and private key of a TLS enabled service. You must keep this file private, and not distribute it or store the file in a non-secure location such as a git repository. The Web API Server's keystore file should be unique per node, with a certificate that is signed (issued) by a trusted CA.

The default self-signed key pair for the Web API Server is generated when installing the iris-config.rpm and saved to .p12 keystore file:

[-r-------- irisadmin irisadmin] webServices-keystore.p12

The keystore file is protected by a unique randomly generated password stored in base64 encoded format in a read-only hidden file owned by user irisadmin with permission set to 400:

[-r-------- irisadmin irisadmin] .webapi_passphrase

When certificates, keys, or truststores are not configured correctly, then connections will fail and additional debug parameters should be set.

Properties file sections

These file locations are specified by the following properties, which need to be accessible in the web_api_service context.

[service.name=web_api_service] {
    WebServer.tls.keystore=/etc/sysconfig/illumon.d/auth/webServices-keystore.p12
    WebServer.tls.passphrase.file=/db/TempFiles/irisadmin/.webapi_passphrase
}

TLS Truststore

A TLS truststore file contains the root certificates that issued any certificates that should be trusted. The Deephaven truststore should contain the root certificate or certificates for all the Authentication Server and Configuration Server certificates. This file is stored in two formats; a PKCS12 file verified by a password, and a PEM file which does not require a password.

All nodes and all clients need to have access to the truststore files. A single truststore should be used across all Deephaven nodes. If your organization has an existing PKI, then you can import your CA's certificate into the truststores for use with Deephaven.

The truststore does not contain secrets, and all clients need access to the password for the password-protected file.

Truststore filenames

• /etc/sysconfig/illumon.d/resources/truststore-iris.p12

• /etc/sysconfig/illumon.d/resources/truststore-iris.pem

Passphrase file

/etc/sysconfig/illumon.d/resources/truststore_passphrase

Properties

• configuration.server.cacertfile=/etc/sysconfig/illumon.d/resources/truststore-iris.pem
• tls.truststore=/etc/sysconfig/illumon.d/resources/truststore-iris.p12
• tls.truststore.passphrase.file=/etc/sysconfig/illumon.d/resources/truststore_passphrase

When certificates, keys, or truststores are not configured correctly, then connections will fail and additional debug parameters should be set.

Using non-default certificates

Instead of using the uniquely generated files from the installation, customers can create their own keystore files with private keys and certificates signed by their own CA and update these properties accordingly.

Import your own root or intermediate CA certificate into the trust store:

    /usr/lib/jvm/java-openjdk/bin/keytool \
    -import -trustcacerts -alias iris \
    -file /path/to/tls.crt -keystore \
    /etc/sysconfig/illumon.d/resources/truststore-iris.p12 \
-storepass $PASS_PHRASE

When certificates, keys, or truststores are not configured correctly, then connections will fail and additional debug parameters should be set.

Authorized keys

A Deephaven-authorized keys file associates usernames with keys and allows non-interactive processes to authenticate with a key instead of a password. The process of authenticating must have the private key corresponding to the public key in this file. This file is disabled by default, with public keys maintained in the ACL store (etcd or MySQL).

As of the Grizzly (1.20240517), Deephaven uses the secp256r1 Elliptic Curve to generate key pairs, which is currently recommended by FIPS.186-5. Deephaven's authentication server generates a random nonce, which the client then hashes using SHA256 and signs with the private key. In versions of Deephaven from 1.20221001 to 1.20231218, Deephaven generated 2048-bit DSA keys as defined in FIPS.186-4. Prior to 1.20221001, 512-bit DSA keys were generated. For backward compatibility, Deephaven supports using ECDSA and DSA keys for authentication so that users can continue using their existing keys.

The authorized keys file contains one public key per line. Lines beginning with # are comments. Each line consists of a user name and a base64-encoded key separated by white space. The public key is encoded in DER format. If the key is an Elliptic Curve key, then it is prefaced with EC:. Other keys are assumed to be DSA keys. The same key encoding is used for public keys stored in the ACL database.

DSA Keys File

/etc/sysconfig/illumon.d/resources/dsakeys.txt

Properties file sections

[service.name=authentication_server] {
    authentication.server.authorizedkeys.enabled=true
    authentication.server.authorizedkeys.file=dsakeys.txt
}

User private keyfile

Users may authenticate using private key files. These should be only on the user's client machine, and provide access to Deephaven just as the user's password. The private key file is specified as follows.

The file is structured as a dictionary. Lines beginning with # are comments. Each line is a key and value separated by white space. There are four fields defined:

Private key files can be generated by using:

• /usr/illumon/latest/bin/generate-iris-keys from the command line.
• io.deephaven.enterprise.auth.KeyPairFile#generateNewKeys from Java/Groovy.
• deephaven_enterprise.client.auth.AuthClient.generate_keypair from Python.

Property

AuthenticationClientManager.defaultPrivateKeyFile

The public key in the named file must be entered in the server’s Authorized Keys file.

Deephaven private keyfile

The controller uses a key to authenticate to the Deephaven remote query dispatchers (query servers and merge servers). The controller must operate as a privileged user because it performs privileged operations such as starting persistent queries owned by any user. This key file should be deployed only to the controller, and not readable by other users.

Property

iris.authentication.keyfile=/etc/sysconfig/illumon.d/auth/priv-iris.base64.txt

Controller key pair

The Deephaven controller may store secrets such as JDBC passwords in its configuration. Consoles must be able to update these secrets, and to store them persistently in the Persistent Query Controller's cache files. This file should be only on the controller node. When a client needs to send a secret to the controller, it requests the public key. The controller decrypts secret data (e.g., passwords) using its private key before sending the secret to workers that require it.

Property

[service.name=iris_controller] {
    PersistentQueryController.keyPairFile=/etc/sysconfig/illumon.d/auth/priv-controllerConsole.base64.txt
}

Other key files

The following key files are also installed.

• priv-authreconnect.base64.txt - This is the keyfile used by the authentication server for its public/private encryption. The file is defined by the following property, which can be updated if needed:
  • authentication.server.reconnection.keyfile=/etc/sysconfig/illumon.d/auth/priv-authreconnect.base64.txt
• priv-merge.base64.txt - This keyfile may be used for authentication by batch merge queries.
• priv-superuser.base64.txt - This keyfile may be used by the superuser user for authentication.

Database authentication

To connect to a database server to run queries, you must authenticate your connection. For interactive utilities, like the Deephaven console, you do so through a username/password dialog box or SAML. For automated batch queries, you must use a private key file. Best practices dictate that system batch jobs are run as a role user. If you are writing your own batch queries (or interactive utilities), you need to provide a method for authenticating your client before any database connection is made. The database components all use the AuthenticationClientManager.getDefault() instance, so authenticating should be performed using one of the following methods:

1. Setting the AuthenticationClientManager.defaultPrivateKeyFile property, which must specify a private key file that is known to the authentication server. Authentication will be done automatically using the specified private key. For a batch query, this is the preferred choice.
2. Calling AuthenticationManager.getDefault().challengeResponse(String privateKeyFile), with a private key file that is known to the authentication server.
3. Implementing a method to get the password from the user. You should not have users store passwords in a file. If the password is typed in, you should make it obscured (for example, by using java.io.Console.readPassword).

How to create Deephaven Authentication Keys

Deephaven can use challenge-response authentication with DSA keys to authenticate users. This is useful for batch or other non-interactive programs that cannot prompt for passwords. DSA keys consist of two parts: (1) a private key that must be protected as if it were a password, and (2) a public key stored in the Deephaven ACL database.

You can generate a public/private key pair for yourself using the Core+ Python client and upload it to the ACL write server. This has the advantage of the private key never leaving the user's machine. Alternatively, a member of iris-acleditors can upload a key for any user.

from deephaven_enterprise.client.session_manager import SessionManager
import deephaven_enterprise.client.auth
import os

# generate a public/private key pair for "user_name"
(priv, pub) = deephaven_enterprise.client.auth.AuthClient.generate_keypair("user_name")

# Write the private key to a file, take care to ensure the permissions are safe.  This key is equivalent to a password.
with open("/tmp/priv.txt", "w") as f:
    f.write(priv)

print("Generated Key")

# Connect to the Deephaven cluster
sm = SessionManager("https://deephaven-cluster.example.com:8123/iris/connection.json")
# You can authenticate using your password, saml, or an existing private key.
sm.password("user_name", "password")
# Upload the public key to the ACL Write Server
sm.upload_key(pub)
sm.close()


# Now reconnect with your private key
sm2 = SessionManager("https://deephaven-cluster.example.com:8123/iris/connection.json")
sm2.private_key("/tmp/priv.txt")
sm2.close()

Alternatively, you can generate a public/private key pair on a Deephaven server as follows:

1. On a Deephaven server, run: /usr/illumon/latest/bin/generate-iris-keys <iris user name>. The default key length is 2048 bits. You can override that with a second parameter:

   /usr/illumon/latest/bin/generate-iris-keys <iris user name> <bits>
   

2. This will generate pub-<iris user name>.base64.txt and priv-<iris user name>.base64.txt in the current directory.

3. Store the generated public key in the ACL database using the command:

   /usr/illumon/latest/bin/dhconfig acl publickey import -f <path to key>/pub-<iris user name>.base64.txt
   

   If you are not running as an administrator, then you should also include the --user parameter to specify which Deephaven account to authenticate as. You can either specify an administrator account or your own account, and dhconfig prompts for your password:

   /usr/illumon/latest/bin/dhconfig acl publickey import -f <public key file> --user <user>
   

4. Move priv-<iris user name>.base64.txt to the client system connecting to the server. This file is effectively a password. It should be stored only where it will be used and must be protected by appropriate filesystem permissions.

How to connect to a DB Server

To connect to a database server to run queries, you must authenticate your connection. For interactive utilities, like the Deephaven console, you do so through a username/password dialog box. For automated batch queries, you must use a private key file. You should generate your own private key that allows you to authenticate to the database for batch jobs like back tests, without having to enter a password, and then provide the public key to the system administrator. Move this file to the machine that you use to run batch jobs. When you run a batch job, make sure to add the JVM property:

AuthenticationClientManager.defaultPrivateKeyFile=<path/to/private/key/file>

• For example, you could use the following property on Windows: AuthenticationClientManager.defaultPrivateKeyFile=C:\\Users\\<username>\\priv-<user>.base64.txt
• Or the following property on Linux: AuthenticationClientManager.defaultPrivateKeyFile=/Users/<username>/priv-{user}.base64.txt

You can set this property from any prop file that you include, or if you prefer, you can also set it from the IntelliJ VM Options by prefacing it with –D. You can do this for an individual configuration, by selecting the configuration and adding -DAuthenticationClientManager.defaultPrivateKeyFile=<path/to/private/key/file> to that configuration's VM Options string.

You can alternatively update the default options for applications to include this property ("Run" menu -> "Edit Configurations" -> "Defaults" -> "Application", "VM Options" field), which will take effect for new configurations (but not your existing configurations).

You should treat this file like a password; other users should not be allowed to read it:

chmod 600 <path/to/private/key/file>

This will allow only the owner to read the file, which can be verified via:

ls -l <path/to/private/key/file>

Permissions should read rw- --- ---.

Related documentation

• Deephaven properties files
• ACLs