How to set up a Deephaven OpenAPI application in Node.js

What is Deephaven OpenAPI?

Deephaven is a powerful data analytics tool that can quickly query and update billions of rows of live data. Node.js is a robust server-side JavaScript engine that can act as a compatibility layer between several different tools. Deephaven OpenAPI lets you easily combine the two by providing an access layer to your Deephaven instance. OpenAPI currently supports JavaScript (with other languages planned), but using it with Node.js requires a few small tweaks. In this guide, we’ll cover how to set up a basic Deephaven OpenAPI application in Node.js.

Polyfill a few missing parts

Since OpenAPI was primarily designed with browser use in mind, there are a few things missing in Node that must be polyfilled.

• First, we need a websocket package. We will use ws.
• Then, we need to add a few events and other global browser objects that are used by the API but are missing in Node.

The following code shows the polyfill which we will call openapiPolyfill.js. Note the location defined should point to your server.

// Set a few objects that do not exist in the Node environment
class Event {
  constructor(type, dict) {
    this.type = type;
    if (dict) {
      this.detail = dict.detail;
    }
  }
}

class CustomEvent extends Event {
  constructor(...args) {
    super(...args);
  }
}

global.self = global;
global.window = global;
global.Event = Event;
global.CustomEvent = CustomEvent;
global.WebSocket = require('ws');

Staying in sync with the server

OpenAPI has some nice helper classes available with the Deephaven release files. These helper classes all reside in a file called ‘irisapi.nocache.js’ on the server. This file is versioned so it must be updated any time the server updates.

To simplify this, we’ll create a function to asynchronously include our API file before we start our app.

The code below will:

• retrieve the required file from your server URL,
• write it to disk,
• require the file,
• and finally resolve a promise to indicate the API is ready to use.

We’ll call this file openapiAsyncInclude.js.

const https = require('https');
const fs = require('fs');

// Required for its side-effects (exposing missing values to global)
require('./openapiPolyfill.js');

module.exports = (url, fileName = 'irisapi.nocache.js') => {
  global.location = new URL(url);
  return new Promise((resolve, reject) => {
    const apiFile = fs.createWriteStream(`./${fileName}`);
    const request = https
      .get(url, (response) => {
        // Write JS text to file
        response.pipe(apiFile);
        apiFile.on('finish', () => {
          apiFile.close(() => {
            require(`./${fileName}`); // require is synchronous, so the resolve won't happen until it is done
            resolve(iris); // iris is globally exposed by the require, but we'll return it anyway
          });
        });
      })
      .on('error', (err) => {
        // Delete the created file and reject when done
        fs.unlink(`./${fileName}`, () => reject(err));
      });
  });
};

Connecting to Deephaven

Now that we’ve got some Node specifics out of the way, we can use OpenAPI to connect to Deephaven. We’ll use our function from the previous section to include the latest API file from the server, and once that is done, we can connect using the Client class on the globally exposed iris object.

The Client class has several events that are triggered when the connection state changes:

• The connect event lets us know we initially connected and can login. After logging in, we can fully use OpenAPI.
• The reconnect event lets us know that our client reconnected, but is still authenticated. One of the nice things about the Client class is that it manages reauthentication for you. If for some reason it is unable to reauthenticate, then you get the reconnectauthfailed event. It may be useful to listen for this event and attempt to restart our application. Maybe a server update caused our files to be out of sync which caused the authentication failure. Thankfully, you can require files at any point in Node which makes this a logical place to download the latest API file from the server and restart.
• The final event of interest is disconnect. This event indicates exactly what it says: our client disconnected from the server. You could add a delay and then try to restart the app. You could also send some sort of notification so you know a problem occurred.

The code below is an example of how to put everything together so you can start using Deephaven OpenAPI:

const irisInclude = require('./openapiIncludeAsync');

const host = 'server.com';
const credentials = { username: 'user', token: 'pass', type: 'password' };
const apiFileName = 'irisapi.nocache.js';
let client;

start();
async function start() {
  try {
    await irisInclude(`https://${host}/irisapi/${apiFileName}`, apiFileName);
    const wsUrl = `wss://${host}/socket`;

    client = new iris.Client(wsUrl);
    client.addEventListener(iris.Client.EVENT_CONNECT, async () => {
      await client.login(credentials);
      console.log('logged in');
      // Do things with Deephaven!
    });

    client.addEventListener(iris.Client.EVENT_RECONNECT, async () => {
      console.log('reconnected and already authenticated');
    });

    client.addEventListener(
      iris.Client.EVENT_RECONNECT_AUTH_FAILED,
      async () => {
        console.log('need to reauth');
        start();
      }
    );

    client.addEventListener(iris.Client.EVENT_DISCONNECT, () => {
      console.log('disconnected');
      setTimeout(start, 30000);
    });
  } catch (err) {
    // Couldn’t connect
    console.error(err);
  }
}