Authentication Server runbook

The Authentication Server is a core Deephaven infrastructure service responsible for authenticating users and issuing authentication tokens. It validates credentials through multiple authentication methods. See the Authentication Service overview for a full description of services provided.

Impact of Authentication Server failure

Level	Impact
Sev 1 - Critical	New users will be unable to log in or create new queries.

Caution

Users with valid, non-expired authentication tokens can continue working during an Authentication Server outage, but new logins and token refreshes will fail.

Authentication Server dependencies

The Authentication Server requires:

etcd cluster — Must be running and accessible for ACL storage (if using etcd for ACLs).
MySQL/MariaDB — Must be running if using MySQL for ACL storage.
etcd client configuration files — Located at /etc/sysconfig/deephaven/etcd/client.
LDAP/Active Directory — Must be accessible if using LDAP authentication.
SAML Identity Provider — Must be accessible if using SAML authentication.

The Authentication Server does not depend on the Configuration Server, allowing it to start independently.

Supported authentication methods

The Authentication Server supports multiple authentication methods:

Built-in username/password — Stored in ACL database (etcd or MySQL).
LDAP/Active Directory — Validates credentials against directory service.
SAML 2.0 — Integrates with enterprise identity providers.
DSA keys — Public key authentication for programmatic access.

Configuring the Authentication Server

ACL backend configuration:

For etcd-based ACLs:

acls.dataSource=etcd

For MySQL-based ACLs:

acls.dataSource=mysql
db.url=jdbc:mariadb://localhost:3306/dbacl_iris
db.user=iris
db.password=<encrypted-password>

Caution

TLS should never be disabled unless you have very strong network isolation and fully trust all users with access to your network.

Checking Authentication Server status

Check process is running with monit:

dh_monit status authentication_server

Expected output should show status Running.

Test authentication endpoint connectivity:

# Test if server is responding (adjust hostname and port as needed)
# Use --insecure if the server uses a self-signed certificate
grpcurl --insecure localhost:22123 list

Viewing Authentication Server logs

View application log:

cat /var/log/deephaven/authentication_server/AuthenticationServer.log.current

Tail the log to follow in real-time:

tail -f /var/log/deephaven/authentication_server/AuthenticationServer.log.current

List historical log files:

ls -ltr /var/log/deephaven/authentication_server/authentication_server.log.????-??-??

View process stdout/stderr logs:

cat /var/log/deephaven/authentication_server/authentication_server.log.$(date +%Y-%m-%d)

Restart procedure

Restart the Authentication Server:

dh_monit restart authentication_server

Verify the restart was successful:

dh_monit status authentication_server

Monitor the log during startup:

tail -f /var/log/deephaven/authentication_server/AuthenticationServer.log.current

Expected startup messages:

Successful connection to etcd or MySQL.
Server listening on configured port.
No authentication backend errors.

Checking dependencies

Check etcd status

If using etcd for ACLs:

sudo -u irisadmin /usr/illumon/latest/bin/etcdctl.sh endpoint status --write-out table

Check MySQL status

If using MySQL for ACLs:

sudo systemctl status mariadb

Test MySQL connectivity:

sudo mysql -e "SELECT COUNT(*) FROM users" dbacl_iris

Authentication modes and ACL integration

Built-in authentication

Users and password hashes are stored directly in the ACL database. Managed through the ACL Write Server or dhconfig acls commands.

LDAP authentication

Validates credentials against LDAP/Active Directory. Users must still exist in the ACL database for authorization. See LDAP configuration for setup and the full property reference.

SAML authentication

Delegates authentication to an enterprise identity provider (IdP). See SAML configuration for setup and the full property reference.

DSA key authentication

Public key authentication for programmatic API access. The dhconfig acls publickeys subcommand manages the public keys used for DSA key authentication. See dhconfig acls command reference for usage.

Configuration files and locations

monit configuration: /etc/sysconfig/illumon.d/monit/authentication_server.conf

Property files:

/etc/sysconfig/illumon.d/resources/iris-common.prop
/etc/sysconfig/illumon.d/resources/authentication_server.prop

TLS keystore: /etc/sysconfig/deephaven/auth/keystore.authserver.p12

Keystore passphrase: /etc/sysconfig/deephaven/auth/.auth_passphrase

Log directory: /var/log/deephaven/authentication_server/