--- id: runbook-web-api-server title: Web API Server runbook --- The Web API Server is a critical Deephaven infrastructure service that serves the Deephaven web application UI, brokers worker connections, and hosts the Client Update Service (CUS) for delivering client libraries and resources. It provides the primary user-facing interface for accessing Deephaven. ## Impact of Web API Server failure | Level | Impact | | :--------------- | :----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | Sev 1 - Critical | Both Web API clients and Deephaven Console GUI Users will be impacted. Users will not be able to use the Launcher and Deephaven Clients will not be able to receive any updates from the server. | > [!CAUTION] > The Web API Server does not proxy data connections. After the initial handshake, clients connect directly to worker processes. This means existing sessions can continue working briefly even if the Web API Server fails, but new sessions cannot be created. ## Web API Server dependencies The Web API Server requires: 1. **Configuration Server** — Must be running to access cluster configuration. 2. **Authentication Server** — Must be running to validate user logins. 3. **Remote Query Dispatcher** — Must be accessible to allocate worker processes. 4. **etcd cluster** — Must be accessible (via Configuration Server). **Optional dependencies:** - **Envoy** — If using Envoy, the Web API Server may register with it. ## Checking Web API Server status Check process is running with monit: ```bash dh_monit status web_api_service ``` Expected output should show status `Running`. Test web UI accessibility: ```bash # Check if web server is responding curl -k https://localhost/ ``` ## Viewing Web API Server logs View application log: ```bash cat /var/log/deephaven/web_api_service/WebServer.log.current ``` Tail the log to follow in real-time: ```bash tail -f /var/log/deephaven/web_api_service/WebServer.log.current ``` List historical log files: ```bash ls -ltr /var/log/deephaven/web_api_service/web_api_service.log.????-??-?? ``` View process stdout/stderr logs: ```bash cat /var/log/deephaven/web_api_service/web_api_service.log.$(date +%Y-%m-%d) ``` ## Restart procedure Restart the Web API Server: ```bash dh_monit restart web_api_service ``` **Impact:** Restarting the Web API Server will: - Disconnect users from the web UI (they'll need to refresh their browsers). - Interrupt the initial connection handshake for new sessions. - Temporarily prevent access to the Client Update Service. Existing worker sessions will continue running but users may lose their browser connection. Verify the restart was successful: ```bash dh_monit status web_api_service ``` Monitor the log during startup: ```bash tail -f /var/log/deephaven/web_api_service/WebServer.log.current ``` ## Enabling Web API Service The Web API Service is disabled by default in some configurations. To enable it: 1. Remove the `.disabled` extension from the config file: ```bash cd /etc/sysconfig/illumon.d/monit mv web_api_service.conf.disabled web_api_service.conf ``` 2. Reload monit configuration: ```bash dh_monit reload ``` 3. Start the service: ```bash dh_monit start web_api_service ``` ## Configuring the Web API Server Key configuration properties (typically in `iris-common.prop` or a service-specific property file): | Property Name | Purpose | Default Value | | :------------------------------------- | :----------------------------------------- | :------------------------------------------------------- | | `WebServer.tls.keystore` | Path to TLS keystore (p12 format) | `/etc/sysconfig/illumon.d/auth/webServices-keystore.p12` | | `WebServer.tls.passphrase.file` | Path to keystore password file | `/db/TempFiles/irisadmin/.webapi_passphrase` | | `RemoteQueryDispatcherParameters.host` | Hostname clients use to connect to workers | Server FQDN | ## TLS certificate management The Web API Server uses TLS to provide secure HTTPS connections. ### Default self-signed certificate A default self-signed certificate is generated during installation: **Keystore location:** `/etc/sysconfig/illumon.d/auth/webServices-keystore.p12` **Keystore password:** Stored in `/db/TempFiles/irisadmin/.webapi_passphrase` **Permissions:** ```bash -r-------- irisadmin irisadmin webServices-keystore.p12 -r-------- irisadmin irisadmin .webapi_passphrase ``` > [!CAUTION] > The default self-signed certificate works for testing but presents security warnings in browsers. Always use a CA-signed certificate for production. ### Installing a CA-signed certificate 1. Obtain a TLS certificate signed by your trusted CA with the domain name matching your Deephaven server (e.g., `myserver.mydomain.com`). 2. Backup the existing keystore: ```bash sudo cp /etc/sysconfig/illumon.d/auth/webServices-keystore.p12 \ /etc/sysconfig/illumon.d/auth/webServices-keystore.p12.ORG ``` 3. Import your CA cert and key files to the keystore: ```bash STOREPASS=$(sudo cat /db/TempFiles/irisadmin/.webapi_passphrase) # This assumes you have stored your .key and .crt in /etc/ssl/certs/ openssl pkcs12 -export -in /etc/ssl/certs/tls.crt -inkey /etc/ssl/certs/tls.key \ -name webapi -out /etc/sysconfig/illumon.d/auth/webServices-keystore.p12 \ -passout pass:$STOREPASS ``` 4. Set correct permissions: ```bash sudo chown irisadmin:irisadmin /etc/sysconfig/illumon.d/auth/webServices-keystore.p12 sudo chmod 400 /etc/sysconfig/illumon.d/auth/webServices-keystore.p12 ``` 5. Update dispatcher hostname to match certificate: Edit `/etc/sysconfig/illumon.d/resources/iris-common.prop`: ```properties RemoteQueryDispatcherParameters.host=myserver.mydomain.com ``` 6. Restart Web API Service: ```bash dh_monit restart web_api_service ``` 7. Restart Query Dispatchers to pick up hostname change: ```bash dh_monit restart db_query_server dh_monit restart db_merge_server ``` ## Client Update Service The Client Update Service (CUS) is hosted by the Web API Server and provides client-side components to remote clients. ### Accessing the Client Update Service The CUS is available at: ``` https:///launcher ``` Clients can download installers for: - Windows desktop launcher. - Mac desktop launcher. - Linux desktop launcher. ### CUS-delivered resources The CUS delivers: - Client JARs (Java libraries for remote clients). - Property files (client-side configuration). - Business calendars. - UI themes. - Custom client configurations. These files are sourced from: - `/usr/illumon/latest/lib/` (JARs). - `/etc/sysconfig/illumon.d/resources/` (properties). - `/etc/sysconfig/illumon.d/calendars/` (business calendars). ### Reloading the Client Update Service When server components change (new JARs, updated properties, etc.), reload the CUS: Navigate to: ``` https:///reload/ ``` Or programmatically: ```bash curl -k https:///reload/ ``` > [!NOTE] > The CUS reload does NOT refresh properties. If properties affecting CUS configuration have changed, you must restart the Web API Service. **After CUS reload:** - Clients must exit and restart their launcher to download new components. - Clients that don't restart may have outdated code incompatible with the server. ## Configuration files and locations **monit configuration:** `/etc/sysconfig/illumon.d/monit/web_api_service.conf` **Property files:** - `/etc/sysconfig/illumon.d/resources/iris-common.prop` - `/etc/sysconfig/illumon.d/resources/web_api_service.prop` **TLS keystore:** `/etc/sysconfig/illumon.d/auth/webServices-keystore.p12` **Keystore passphrase:** `/db/TempFiles/irisadmin/.webapi_passphrase` **Log directory:** `/var/log/deephaven/web_api_service/` **CUS source directories:** - `/usr/illumon/latest/lib/` - `/etc/sysconfig/illumon.d/resources/` - `/etc/sysconfig/illumon.d/calendars/` ## Related documentation - [System processes overview](../architecture/architecture-overview.md) - [TLS certificates](../configuration/pki-certificate.md) - [Launcher and client configuration](../../legacy/legacy-ui/launcher-and-client-configuration.md) - [Remote Query Dispatcher runbook](runbook-remote-query-dispatcher.md) - [Configuration Server runbook](runbook-config-server.md)