Connect Solace

The Deephaven engine can directly consume data from a Solace PubSub+ Mesh™. By doing so, you can improve latency and throughput relative to other solutions such as having an external logger write data to a binary log file. All you need to enable this integration is to install the Solace integration plugin and do some straightforward system setup to tell Deephaven where your Solace-based data should be saved on disk.

The Solace integration plugin exposes new classes that can be invoked from within Persistent Queries. This way, the only system-level configuration you need to do is telling the system where to save your data. The details of how to connect to Solace, what data you'll retrieve, and how to parse that data into the columns of your Deephaven table are all set up within your Persistent Query. This way, you can use all of the options that Solace offers without your Deephaven administrator needing to pre-configure any of them.

The two primary options offered by the Solace integration plugin are Queue-based ingestion and Topic-based ingestion. These have different powerful uses, depending on what kind of data you're interested in.

Queue-based ingestion is best if you have data that needs to be delivered in toto, in order. A Solace Queue can be set up to persist data until it's consumed, so if your Deephaven system didn't get turned on until later in the day, your Deephaven system could still retrieve information from earlier. For example, if you had an inventory management system based on Deephaven, you would want to know all of your transactions for the day so as to be able to calculate your current holdings. Queue-based ingestion can be replicated across multiple servers and works with redundant delivery for high availability.

Topic-based ingestion is best when you have data that needs to be delivered in best-effort, as-available form. A Solace Topic that is not persisted will deliver data on the fly, but does not retain that data for later ingestion. If your Deephaven system is not running when a particular piece of data is published to a Solace Topic, that piece of data will not be available to Deephaven. For example, if you had an inventory management system based on Deephaven, you might want to know the current market price of each asset in your system, but not need anything other than the current price. Topic-based ingestion is single-server (since different servers cannot be sure they started by receiving the same message) and is best for transient data.

For step-by-step instructions and an example Persistent Query, see the Solace Integration Plugin.