Web API Server runbook

The Web API Server is a critical Deephaven infrastructure service that serves the Deephaven web application UI, brokers worker connections, and hosts the Client Update Service (CUS) for delivering client libraries and resources. It provides the primary user-facing interface for accessing Deephaven.

Impact of Web API Server failure

LevelImpact
Sev 1 - CriticalBoth Web API clients and Deephaven Console GUI Users will be impacted. Users will not be able to use the Launcher and Deephaven Clients will not be able to receive any updates from the server.

Caution

The Web API Server does not proxy data connections. After the initial handshake, clients connect directly to worker processes. This means existing sessions can continue working briefly even if the Web API Server fails, but new sessions cannot be created.

Web API Server dependencies

The Web API Server requires:

  1. Configuration Server — Must be running to access cluster configuration.
  2. Authentication Server — Must be running to validate user logins.
  3. Remote Query Dispatcher — Must be accessible to allocate worker processes.
  4. etcd cluster — Must be accessible (via Configuration Server).

Optional dependencies:

  • Envoy — If using Envoy, the Web API Server may register with it.

Checking Web API Server status

Check process is running with monit:

Expected output should show status Running.

Test web UI accessibility:

Viewing Web API Server logs

View application log:

Tail the log to follow in real-time:

List historical log files:

View process stdout/stderr logs:

Restart procedure

Restart the Web API Server:

Impact: Restarting the Web API Server will:

  • Disconnect users from the web UI (they'll need to refresh their browsers).
  • Interrupt the initial connection handshake for new sessions.
  • Temporarily prevent access to the Client Update Service.

Existing worker sessions will continue running but users may lose their browser connection.

Verify the restart was successful:

Monitor the log during startup:

Enabling Web API Service

The Web API Service is disabled by default in some configurations.

To enable it:

  1. Remove the .disabled extension from the config file:
  1. Reload monit configuration:
  1. Start the service:

Configuring the Web API Server

Key configuration properties (typically in iris-common.prop or a service-specific property file):

Property NamePurposeDefault Value
WebServer.tls.keystorePath to TLS keystore (p12 format)/etc/sysconfig/illumon.d/auth/webServices-keystore.p12
WebServer.tls.passphrase.filePath to keystore password file/db/TempFiles/irisadmin/.webapi_passphrase
RemoteQueryDispatcherParameters.hostHostname clients use to connect to workersServer FQDN

TLS certificate management

The Web API Server uses TLS to provide secure HTTPS connections.

Default self-signed certificate

A default self-signed certificate is generated during installation:

Keystore location: /etc/sysconfig/illumon.d/auth/webServices-keystore.p12

Keystore password: Stored in /db/TempFiles/irisadmin/.webapi_passphrase

Permissions:

Caution

The default self-signed certificate works for testing but presents security warnings in browsers. Always use a CA-signed certificate for production.

Installing a CA-signed certificate

  1. Obtain a TLS certificate signed by your trusted CA with the domain name matching your Deephaven server (e.g., myserver.mydomain.com).

  2. Backup the existing keystore:

  1. Import your CA cert and key files to the keystore:
  1. Set correct permissions:
  1. Update dispatcher hostname to match certificate:

Edit /etc/sysconfig/illumon.d/resources/iris-common.prop:

  1. Restart Web API Service:
  1. Restart Query Dispatchers to pick up hostname change:

Client Update Service

The Client Update Service (CUS) is hosted by the Web API Server and provides client-side components to remote clients.

Accessing the Client Update Service

The CUS is available at:

Clients can download installers for:

  • Windows desktop launcher.
  • Mac desktop launcher.
  • Linux desktop launcher.

CUS-delivered resources

The CUS delivers:

  • Client JARs (Java libraries for remote clients).
  • Property files (client-side configuration).
  • Business calendars.
  • UI themes.
  • Custom client configurations.

These files are sourced from:

  • /usr/illumon/latest/lib/ (JARs).
  • /etc/sysconfig/illumon.d/resources/ (properties).
  • /etc/sysconfig/illumon.d/calendars/ (business calendars).

Reloading the Client Update Service

When server components change (new JARs, updated properties, etc.), reload the CUS:

Navigate to:

Or programmatically:

Note

The CUS reload does NOT refresh properties. If properties affecting CUS configuration have changed, you must restart the Web API Service.

After CUS reload:

  • Clients must exit and restart their launcher to download new components.
  • Clients that don't restart may have outdated code incompatible with the server.

Configuration files and locations

monit configuration: /etc/sysconfig/illumon.d/monit/web_api_service.conf

Property files:

  • /etc/sysconfig/illumon.d/resources/iris-common.prop
  • /etc/sysconfig/illumon.d/resources/web_api_service.prop

TLS keystore: /etc/sysconfig/illumon.d/auth/webServices-keystore.p12

Keystore passphrase: /db/TempFiles/irisadmin/.webapi_passphrase

Log directory: /var/log/deephaven/web_api_service/

CUS source directories:

  • /usr/illumon/latest/lib/
  • /etc/sysconfig/illumon.d/resources/
  • /etc/sysconfig/illumon.d/calendars/