Core+ C++ Client

The Deephaven Core+ C++ client allows you to create and connect to Persistent Queries using the Community Core engine in the Deephaven Enterprise system. This page provides a brief overview of using the C++ client.

Obtain the C++ client sources

The C++ client is distributed as a source code bundle. This bundle includes a dockerized build script that can be used in any Linux machine with Docker and Docker's buildx installed (buildx is the Docker CLI plugin for extended build capabilities with BuildKit). The script can produce a binary package for supported plaforms (at this time, x86_64 for Ubuntu 22.04, Fedora 38, and RHEL 8, more to come).

The bundle also includes several code examples in the cpp-client/examples directory.

You can request the latest version of the source bundle from your Deephaven Enterprise representative.

Basic Usage

Create a SessionManager

A SessionManager object allows a client to keep connections to a Deephaven Enterprise Controller and Authentication services. First, we need to create a SessionManager object, which will create the connections and keep them alive for as long as the SessionManager object is valid (not destroyed).

#include <string>

#include "deephaven_enterprise/session/session_manager.h"
#include "deephaven_enterprise/utility/utility.h"

using deephaven_enterprise::session::SessionManager;
using deephaven_enterprise::utility::GetTidAsString;

int main(char *argv[], int argc) {
  // ...
  const std::string descriptive_name =
      std::string(basename(argv[0])) +
      " tid=" + GetTidAsString();
  const json_url = "https://server1.mydomain.com:8000/iris/connection.json";
  SessionManager session_manager = SessionManager::FromUrl(descriptive_name, json_url);
  // ...

The SessionManager::FromUrl factory method takes two arguments.

1. A descriptive name that will be used on the server and client sides to log information about operations performed by this client.
2. A URL pointing to a file in JSON format containing information about connectivity parameters for the Deephaven installation. Such a file is provided by a Deephaven installation under iris/connection.json.

The descriptive name, in principle, can be any string of our choosing. However, it is important to select a value that allows us to distinguish our client in logs. It makes sense to include the process and/or thread id where the client was created; the function GetTidAsString() is provided for that purpose.

During the creation of the SessionManager object, three network connections are made in the order indicated below.

1. The connection.json file is downloaded from the provided URL. This is a temporary connection that only lives until we get the file.
2. A connection is made to the Deephaven Enterprise Authentication service.
3. A connection is made to the Deephaven Enterprise Controller service.

The connections to Authentication and Controller are made based on the information downloaded from connection.json. These two connections are maintained for as long as the SessionManager object is valid.

All these three operations need to succeed for SessionManager to be properly initialized or an error results. When experiencing connectivity issues, or if the infomation contained in the connection.json file downloaded is wrong, you may see an error similar to:

Error: std::string deephaven_enterprise::utility::GetUrl(const string&)@/opt/deephaven/src/iris/DhcInDhe/cpp-client/utility/src/net.cc:121: Timeout was reached

You can get more information about Core+ clients in general and their connectivity in the Core+ Clients section of our Communications Protocols documentation.

Authenticate

Once a SessionManager is created, we need to authenticate it. We have two options: using password authentication, providing a user and password, or using private key authentication, providing a private key file.

  bool auth_result = false;
  if (use_password_authentication) {
    // use password authentication
    auth_result =
      session_manager.PasswordAuthentication(user, password, operate_as);
  } else {
    // use private key authentication
    auth_result =
      session_manager.PrivateKeyAuthentication(private_key_filename);
 }

An authenticated SessionManager can be used to connect to an existing Persistent Query (PQ) or to create new PQs. In either case, the result of a connection is a DndClient object.

The DndClient object can be used to send queries to a PQ and get results from it.

Connect to an existing Persistent Query

To connect to an existing PQ, we need to know either the string name or the numeric serial of the PQ we want to connect to.

   const std::string pq_name = "my_favorite_pq";
   DndClient client1 = session_manager.ConnectToPqByName(pq_name);

   using deephaven_enterprise::session::pqserial_t;  // a signed numeric type
   const pqserial_t pq_serial = 9876543210L;
   DndClient client2 = session_manager.ConnectToPqBySerial(pq_serial);

Create a new temporary Persistent Query and connect to it

To create a temporary PQ, we first need to create a PQ configuration. We can use a PqConfigBuilder object for that. The config object comes with predefined defaults for most parameters. Consult the class documentation for PqConfigBuilder for the different options. At a minimum, we have to set the desired heap size for our worker in gigabytes.

  const std::string pq_name = "my_new_pq";
  PqConfigBuilder pq_builder = session_manager.MakeTempPqConfigBuilder(pq_name);
  pq_builder.SetMaxHeapSizeGb(4);
  pq_builder.SetScriptLanguage("groovy");

Once we have a PqConfigBuilder ready with our options, we can call its Build method to obtain the configuration, and pass it to the AddQueryAndConnect method of our SessionManager.

  DndClient client3 = session_manager.AddQueryAndConnect(pq_builder.Build());

Send a query to a DndClient and get the result

To be able to interact with the tables in a PQ from the client we first need to create a DndTableHandleManager object from the DndClient.

  // DndClient client = ...
  DndTableHandleManager table_manager = client.GetManager();

With a DndTableHandleManager on hand, we can run a script on the server (PQ). In the example below, we create a table on the server (PQ) and then fetch that table on the client using a TableHandle object.

A TableHandle object can be used to perform additional operations on the table from the client side. In our example, we just print the table to standard output.

  // "PT1S" is an ISO-8601 duration, it specifies a duration of 1 second.
  table_manager.RunScript(
      "my_table = io.deephaven.engine.util.TableTools.timeTable(\"PT1S\")");
  sleep(2);
  TableHandle table_handle = table_manager.FetchTable("my_table");
  std::cout << "*** Table my_table is \n"
            << table_handle.Stream(true) << "\n";

Close DndClient and SessionManager objects

When DndClient and SessionManager objects are destroyed, they are automatically closed. This closes network connections and releases all server-side resources associated with these objects. You can call the Close method before the object is destroyed to force it to close right away.

  client1.Close();
  client2.Close();
  client3.Close();
  client3.Close();
  session_manager.Close();

Ensuring these objects are closed promptly supports the best utilization of system resources.

Appendix

Connectivity and tools to debug network issues

Deephaven clients use gRPC, an open source RPC framework from Google, to implement communications with Deephaven servers. Part of the nature of how gRPC works implies treating network errors as potentially transient: network failures are based on a timeout model. This implies that the wrong host address or a server that is down does not immediately cause a connection attempt to fail; instead gRPC keeps trying to connect, in the assumption that a new host name registration may appear or a server may restart and make the expected port available.

Eventually, if the network issue is not resolved before a timeout window expires, the connection attempt is considered failed and an error results.

Treating network connectivity issues always as potentially transient failures is very useful in terms of making deployed infrastructure, client and servers resilient. From the point of view of interactive use, however, it results in some opaqueness for users: if there is a network failure, trying to start a client may look like it just 'sits there' for two minutes before raising an error.

Related documentation

• Core+ Clients
• UI Query Management