Remote Query Dispatcher configuration
The Remote Query Dispatcher (simply called the dispatcher) is responsible for creating all Deephaven workers in a Deephaven installation. Each installation will have at least two dispatchers: a query server and a merge server. Like all Deephaven processes, the base property file for any Deephaven process is specified by the Configuration.rootFile property, and property files are retrieved from the configuration server.
- Query servers run most workers that do not require the ability to write data. There is at least one query server in a Deephaven installation. Additional query servers can be configured to increase capacity, typically by running an additional query server on each additional machine.
- Merge servers run privileged workers that are allowed to write to (and delete) disk-backed tables, both intraday and historical. Only users with specific privileges are allowed to start merge workers. Many installations only need one merge server, although more can be added if necessary.
The following configuration options are likely of most interest to administrators:
- Worker heap size options allow administrators to configure and restrict worker heap sizes.
- Dispatcher usage restrictions allow administrators to restrict the usage of dispatchers. For example, it is possible to only allow certain users to start workers on a specific dispatcher. Merge servers automatically restrict usage to users with certain privileges.
A description of the dispatcher's communications protocols may also be of interest.
Updating properties
See viewing and changing configuration for details on how to change property files.
By default, properties apply to all Deephaven processes. Stanzas restrict properties so that they only apply to certain processes, such as one specific dispatcher process.
- Place properties that are required for all dispatchers (query servers and merge servers) in a property file outside any stanzas, and before any dispatcher-specific properties.
- Following this, place properties that are overridden for specific dispatchers within a stanza. For instance, put merge server properties in a
[service.name=dbmerge]stanza.iris_defaults.propfile first defines ports (properties such asRemoteQueryDispatcherParameters.queryPort) outside any stanzas. It then adds the merge-server values within a[service.name=dbmerge]stanza.iris-endpoints.propuses ahost=stanza to restrict properties to one specific dispatcher.
Any property updates require the impacted dispatcher processes to be restarted.
Properties
Dispatcher properties can be broken into several areas:
- Ports changes
- Web server
- Other properties
Ports
The ports that the dispatchers use can be changed if necessary. Query servers and merge servers use different ports so that they can run on the same machine.
| Property Name | Description | Default |
|---|---|---|
RemoteQueryDispatcherParameters.queryPort | The port on which the dispatcher listens for client connections, such as from the Persistent Query Controller (this also requires updating the iris.db.<server number>.port in the controller's server definitions). | 22013 (query server), 30002 (merge server) |
RemoteQueryDispatcher.webserver.port | The port on which the dispatcher's web service listens - workers connect to this port to establish run-time details. | 8084 (query servers), 8085 (merge servers) |
RemoteQueryDispatcher.workerServerPorts | The ports on which the workers started by this dispatcher will listen for client connections. | 23000-23999 (query servers), 25000-25999 (merge servers) |
RemoteQueryDispatcher.workerServerWebsocketPorts | The ports on which the workers started by this dispatcher will listen for web client connections. | 24000-24999 (query servers), 26000-26999 (merge servers) |
Web server
The dispatcher provides an optional web server that can be queried for its current state. This is turned off by default, and can be enabled with the following property:
RemoteQueryDispatcher.webserver.enabled=true
If the web server is enabled, it uses the following properties:
| Property Name | Description | Default |
|---|---|---|
RemoteQueryDispatcher.webserver.sslRequired | If this is defined as true, an SSL certificate will be required and used for the Remote Query Dispatcher's Web services, and the connection will be over https. | true |
RemoteQueryDispatcher.webserver.authenticationRequired | If this is true, users will be required to authenticate to access the Remote Query Dispatcher's Web services. | true |
RemoteQueryDispatcher.tls.keystore | If SSL is being used, the keystore to be searched to find the certificate for the encrypted https connection. | /etc/sysconfig/illumon.d/auth-user/webServices-keystore.p12 |
RemoteQueryDispatcher.tls.passphrase.file | If SSL is being used, a file containing the base64-encoded passphrase used to verify the integrity of the keystore. | /etc/sysconfig/deephaven/auth-user/.webapi_passphrase |
RemoteQueryDispatcher.tls.passphrase | If SSL is being used, the base64-encoded passphrase to be used to verify the integrity of the keystore (RemoteQueryDispatcher.tls.passphrase.file is recommended instead of this property). | None |
If authentication is required, then sslRequired must be true (i.e., credentials are not allowed to be sent over an insecure connection).
Deephaven recommends using passphrase files rather than passphrases. The passphrase file must be protected against reading by non-privileged operating system users.
Note
Do not change the following property, as it is essential for workers to function, even if the web server is not enabled.
RemoteQueryDispatcher.websocket.enabled=true
Other dispatcher properties
| Property Name | Description | Default |
|---|---|---|
RemoteQueryDispatcher.shutdownGracePeriodMS | Optional; when set, the dispatcher should try to terminate all workers and wait up to a specified timeout to accumulate logs before going down when possible. If it is not set then the dispatcher tells the workers to terminate but does not wait for the results. | None |
RemoteQueryDispatcherParameters.name | An optional name used for display and log purposes. | |
RemoteQueryDispatcherParameters.host | The default dispatcher host to which clients will attempt to connect. | 8084 |
RemoteQueryDispatcher.maxConcurrentStartups | The number of workers that a dispatcher will start concurrently. | 10 |
RemoteQueryDispatcher.pushClassGroups | Defines the groups that are allowed to push classes to a server. | * |
RemoteQueryDispatcher.adminGroups | Defines the groups that are allowed to perform administrative dispatcher functions such as terminating workers. | iris-superusers |
Note
Be cautious when modifying RemoteQueryDispatcher.maxConcurrentStartups. Starting workers consumes significant system resources, so thoroughly test any increases to this value. On some servers, higher system utilization can lead to worker startup timeouts.
The dispatcher can be configured to start workers either on a local (bare-metal or VM) server or with Kubernetes pods by using the RemoteQueryDispatcher.workerControlType property. This option is automatically configured by the installer so it should not be updated; it is shown here for reference only.
The values allowed for this property are:
LocalProcess- the dispatcher starts a new process on its server using local scripts. This is the default value.Kubernetes- the dispatcher uses Kubernetes control to start a new pod for each worker.
Here is the default on a Kubernetes installation.
RemoteQueryDispatcher.workerControlType=Kubernetes