Run Deephaven from your own Java application

The Deephaven server is a Java library. The production application and the Docker images are prebuilt applications that start it, but any JVM can start a Deephaven server. This guide shows you how to build your own Java application that starts a Deephaven server, so that you control the classpath, the startup code, and which server components are included.

Build a custom application when you want to:

Package Deephaven together with your own Java code and dependencies in a single application.
Create tables in Java when the server starts and publish them to clients and the web UI.
Replace or remove server components, such as the authorization rules.

If you only need to add JARs to a standard server, you don't need a custom application. See Install and use Java packages instead.

Prerequisites

Java 17.0.5 or later. Deephaven's web server, Jetty 12, requires Java 17. By default, the server doesn't start on earlier Java 17 releases because of a JVM bug (JDK-8287432); if you can't upgrade, the startup error names a JVM workaround.
A build tool that resolves Maven artifacts, such as Gradle or Maven. The examples in this guide use Gradle.
Familiarity with Dagger, the dependency injection framework that assembles the Deephaven server. You need only the basics: components, modules, and the @Provides, @Binds, and @IntoSet annotations.

How a Deephaven server starts

The production application's entry point, io.deephaven.server.jetty.JettyMain, is short. A custom application follows the same three steps:

Call MainHelper.init. It initializes logging, loads the Deephaven configuration, and installs process-wide handlers for shutdown and uncaught errors. Call it before anything else touches Deephaven classes.
Build a Dagger component. The component wires together every server service: the gRPC and Arrow Flight APIs, the web UI, authentication, the console, and Application Mode. A component factory, a subclass of ComponentFactoryBase, builds the component from the configuration.
Get the server from the component and call run, which starts the server and returns once it is listening. Call join to block until the server shuts down.

The production application's component factory, CommunityComponentFactory, belongs to the server-jetty-app project, which isn't published to Maven Central. A custom application defines its own component and factory using the modules in the published deephaven-server-jetty artifact. The rest of this guide walks through how to do that.

Tip

The deephaven-core repository contains a complete example of a custom application in server/jetty-app-custom. It also replaces the authorization provider. If you have a local clone, run it with ./gradlew server-jetty-app-custom:run -Pgroovy.

Set up the build

The following build.gradle file creates an application named my-deephaven-app. Replace <version> with the Deephaven version you want, such as the latest release without its leading v.

plugins {
    id 'application'
}

repositories {
    mavenCentral()
    // Deephaven's Kafka integration depends on artifacts published only to Confluent's repository.
    maven {
        url = 'https://packages.confluent.io/maven/'
        content {
            includeGroup 'io.confluent'
            includeGroup 'org.apache.kafka'
        }
    }
}

def deephavenVersion = '<version>'
def daggerVersion = '<dagger-version>'

dependencies {
    // The Deephaven bill of materials (BOM) keeps every Deephaven artifact on the same version.
    implementation platform("io.deephaven:deephaven-bom:${deephavenVersion}")

    implementation 'io.deephaven:deephaven-server-jetty'

    implementation "com.google.dagger:dagger:${daggerVersion}"
    annotationProcessor "com.google.dagger:dagger-compiler:${daggerVersion}"

    // Logging
    runtimeOnly 'io.deephaven:deephaven-log-to-slf4j'
    runtimeOnly 'io.deephaven:deephaven-logback-print-stream-globals'
    runtimeOnly 'io.deephaven:deephaven-logback-logbuffer'
    runtimeOnly 'ch.qos.logback:logback-classic:<logback-version>'

    // Optional: JVM-specific implementations for memory and GC metrics and a high-resolution clock
    runtimeOnly 'io.deephaven:deephaven-hotspot-impl'
    runtimeOnly 'io.deephaven:deephaven-clock-impl'
}

java {
    toolchain {
        languageVersion = JavaLanguageVersion.of(17)
    }
}

application {
    mainClass = 'com.example.MyServerMain'
    // Pin this so installDist produces a deterministic build/install/my-deephaven-app directory,
    // instead of deriving the name from the Gradle project directory.
    applicationName = 'my-deephaven-app'
    applicationDefaultJvmArgs = [
            '--add-opens', 'java.base/java.nio=ALL-UNNAMED',
            '--add-exports', 'java.management/sun.management=ALL-UNNAMED',
            '--add-exports', 'java.base/jdk.internal.misc=ALL-UNNAMED',
            '-Dio.netty.noUnsafe=false',
            '-Ddeephaven.console.type=groovy',
            '-Ddeephaven.application=my-deephaven-app',
    ]
}

Keep the following in mind:

The Confluent repository is required. deephaven-server-jetty depends on Deephaven's Kafka integration at runtime, and some of its dependencies are published only to Confluent's Maven repository. Without that repository, dependency resolution fails with errors such as Could not find org.apache.kafka:kafka-clients.
Match the Dagger version to Deephaven's. Use the Dagger version listed as a dependency in the deephaven-server POM for your Deephaven version. Deephaven's published JARs contain Dagger-generated code that calls into Dagger's runtime (dagger.internal.*) classes, so a mismatched Dagger version can fail at runtime with a linkage error such as NoSuchMethodError.
Pick a Logback version. Use the Logback version that Deephaven builds and tests against: look up the logback entry in gradle/libs.versions.toml at the tag for your Deephaven version (for example, v<version>). For the Logback configuration file, see Configure logging.
Pass the JVM arguments that apply to your build. The first four are the same arguments that the production application's start script passes. If you launch the application another way, such as from your IDE or a container, pass them there too.
- --add-opens java.base/java.nio=ALL-UNNAMED is always required, because Apache Arrow needs it.
- --add-exports java.management/sun.management=ALL-UNNAMED is required only if you include deephaven-hotspot-impl.
- --add-exports java.base/jdk.internal.misc=ALL-UNNAMED is required only if you include deephaven-clock-impl.
- -Dio.netty.noUnsafe=false is required on Java 25 and later, and harmless on earlier versions.
- -Ddeephaven.application=my-deephaven-app sets the application name used for the data, cache, and config directories; see Configure the server for what it controls and why you should set it.
Pick a console language. The server's default console language is Python, which requires a Python environment that the JVM can load. Setting deephaven.console.type to groovy gives you a Groovy console with no extra setup. Setting it to none disables the console, but only if the component also includes NoConsoleSessionModule; see Write the component factory.

Deephaven's optional integrations are separate artifacts. Add the ones you use as runtime dependencies. For example, the production application also includes deephaven-engine-sql, deephaven-extensions-s3, deephaven-extensions-iceberg-s3, and deephaven-extensions-json-jackson.

Flight SQL needs an extra step, because its artifact doesn't register the service with the server by itself. Add deephaven-extensions-flight-sql as an implementation dependency, not runtimeOnly, so that your code can reference FlightSqlModule. Then include that module in your Dagger component, as described in Write the component factory.

Write the component factory

The component factory builds the Dagger component. Dagger generates the component's implementation at compile time, naming the generated class based on the interface's nesting: MyComponentFactory.MyComponent becomes DaggerMyComponentFactory_MyComponent.

package com.example;

import dagger.Binds;
import dagger.Component;
import dagger.Module;
import dagger.Provides;
import dagger.multibindings.IntoSet;
import io.deephaven.appmode.ApplicationState;
import io.deephaven.client.impl.BarrageSessionFactoryConfig;
import io.deephaven.configuration.Configuration;
import io.deephaven.server.auth.CommunityAuthorizationModule;
import io.deephaven.server.jetty.JettyConfig;
import io.deephaven.server.jetty.JettyServerComponent;
import io.deephaven.server.jetty.JettyServerModule;
import io.deephaven.server.runner.CommunityDefaultsModule;
import io.deephaven.server.runner.ComponentFactoryBase;
import io.deephaven.server.runner.TicketResolversFromServiceLoader;
import io.deephaven.server.session.ClientChannelFactoryModule;
import io.deephaven.server.session.ClientChannelFactoryModule.UserAgent;
import io.deephaven.server.session.SslConfigModule;

import javax.inject.Singleton;
import java.io.PrintStream;
import java.util.List;

public final class MyComponentFactory extends ComponentFactoryBase<MyComponentFactory.MyComponent> {

    @Override
    public MyComponent build(Configuration configuration, PrintStream out, PrintStream err) {
        // Reads http.port, http.host, ssl.* and other web server properties from the configuration.
        final JettyConfig jettyConfig = JettyConfig.buildFromConfig(configuration).build();
        return DaggerMyComponentFactory_MyComponent.builder()
                .withOut(out)
                .withErr(err)
                .withJettyConfig(jettyConfig)
                .build();
    }

    @Singleton
    @Component(modules = MyModule.class)
    public interface MyComponent extends JettyServerComponent {
        @Component.Builder
        interface Builder extends JettyServerComponent.Builder<Builder, MyComponent> {
        }
    }

    @Module(includes = {
            JettyServerModule.class,
            CommunityDefaultsModule.class,
            CommunityAuthorizationModule.class,
            TicketResolversFromServiceLoader.class,
            ClientChannelFactoryModule.class,
            SslConfigModule.class,
    })
    public interface MyModule {

        // Identifies this server when it connects to other Deephaven servers, for example through URIs.
        @Provides
        @UserAgent
        static String providesUserAgent() {
            return BarrageSessionFactoryConfig.userAgent(List.of("my-deephaven-app"));
        }

        // Registers MyApplication, so the server runs it at startup.
        @Binds
        @IntoSet
        ApplicationState.Factory bindsMyApplication(MyApplication app);
    }
}

MyModule also registers MyApplication, the class that publishes tables when the server starts. You write it in Publish tables at startup.

The included modules supply the server:

Module	Provides
`JettyServerModule`	The Jetty web server, which serves the gRPC APIs and the web UI.
`CommunityDefaultsModule`	The standard set of Deephaven services, such as sessions, tables, consoles, plugins, and Application Mode.
`CommunityAuthorizationModule`	The default authorization provider, which lets every authenticated user do everything.
`TicketResolversFromServiceLoader`	Ticket resolvers that plugins on the classpath register through Java's `ServiceLoader`.
`ClientChannelFactoryModule`	Outgoing connections from this server to other Deephaven servers. It requires the `@UserAgent` string.
`SslConfigModule`	The TLS configuration for those outgoing connections.

Warning

CommunityAuthorizationModule binds CommunityAuthorizationProvider, which installs "allow all" authorization wiring for every service it covers: consoles, tables, input tables, storage, partitioned and hierarchical tables, applications, and more. Authentication alone only controls who can connect; with this module included, every authenticated user can do everything on the server. Deployments that need restricted or least-privilege access must replace CommunityAuthorizationModule with their own AuthorizationProvider binding, as described below, rather than relying on authentication alone to restrict access.

To change a part of the server, replace the module that provides it. For example, the jetty-app-custom example leaves out CommunityAuthorizationModule and binds its own AuthorizationProvider, which disables the input table service. Don't include both: Dagger rejects two bindings for the same type.

To add an optional integration that registers its own services, such as Flight SQL, add its module to the @Module(includes = {...}) list. For example, include io.deephaven.server.flightsql.FlightSqlModule from deephaven-extensions-flight-sql to enable Flight SQL, or io.deephaven.server.console.NoConsoleSessionModule from the deephaven-server artifact to support -Ddeephaven.console.type=none, since CommunityDefaultsModule only registers the Python and Groovy console bindings.

Publish tables at startup

An ApplicationState.Factory creates objects when the server starts and exposes them to clients by name. It is the Java equivalent of an Application Mode script. The factory runs with the server's execution context already open.

package com.example;

import io.deephaven.appmode.ApplicationState;
import io.deephaven.engine.context.ExecutionContext;
import io.deephaven.engine.liveness.LivenessScope;
import io.deephaven.engine.liveness.LivenessScopeStack;
import io.deephaven.engine.table.Table;
import io.deephaven.engine.updategraph.UpdateGraph;
import io.deephaven.engine.util.TableTools;
import io.deephaven.util.SafeCloseable;

import javax.inject.Inject;

public final class MyApplication implements ApplicationState.Factory {

    // Keeps the application's tables alive for the life of the server.
    @SuppressWarnings("FieldCanBeLocal")
    private LivenessScope scope;

    @Inject
    public MyApplication() {}

    @Override
    public ApplicationState create(ApplicationState.Listener listener) {
        final ApplicationState state =
                new ApplicationState(listener, MyApplication.class.getName(), "My application");
        final UpdateGraph updateGraph = ExecutionContext.getContext().getUpdateGraph();
        scope = new LivenessScope();
        try (final SafeCloseable ignored = LivenessScopeStack.open(scope, false)) {
            // timeTable creates a refreshing table, and update() on it must run under the update graph's shared lock.
            final Table ticking = updateGraph.sharedLock().computeLocked(
                    () -> TableTools.timeTable("PT1s").update("X = ii"));
            state.setField("ticking", ticking);
            state.setField("staticTable", TableTools.emptyTable(5).update("Y = i * 2"));
        }
        return state;
    }
}

Two details matter here:

Keep the tables alive. Deephaven releases tables that nothing references. Creating the tables inside a liveness scope that the factory holds in a field keeps them alive for as long as the server runs.
Lock the update graph when creating a refreshing table. Some operations on a refreshing table, such as calling update on a time table, must run under the update graph's shared lock. Without it, the server fails to start with IllegalStateException: May not initiate serial table operations. Other operations don't require the lock but work correctly while it's held, so the simplest rule is to create the table under it. Keep the locked block short, because update cycles can't run while you hold the lock. Operations on static tables, such as emptyTable, don't need the lock.

The application's ID is the second constructor argument (after the listener), here com.example.MyApplication. Clients fetch the fields with an application ticket made from the ID and the field name.

Write the main class

The main class follows the three steps from How a Deephaven server starts:

package com.example;

import io.deephaven.configuration.Configuration;
import io.deephaven.server.runner.DeephavenApiServer;
import io.deephaven.server.runner.MainHelper;

public final class MyServerMain {
    public static void main(String[] args) throws Exception {
        // Initialize logging, configuration, and process-wide state. Call this first.
        final Configuration configuration = MainHelper.init(args, MyServerMain.class);

        // Build the Dagger component and start the server. run() returns once the server is listening.
        final DeephavenApiServer server = new MyComponentFactory()
                .build(configuration)
                .getServer()
                .run();

        // Block until the server shuts down.
        server.join();
    }
}

Because run doesn't block, your application can do other work after the server starts, then call join when it has nothing left to do.

MainHelper.init accepts an optional single argument: the path of a properties file whose entries are loaded as system properties before anything else.

Configure logging

Deephaven logs through SLF4J. The build above uses Logback. Add a src/main/resources/logback.xml file that sends logs to the console and to the log buffer that the web UI's Log panel reads:

<configuration>
  <contextListener class="ch.qos.logback.classic.jul.LevelChangePropagator"/>

  <appender name="STDOUT" class="io.deephaven.logback.PrintStreamGlobalsConsole">
    <encoder>
      <pattern>%d{yyyy-MM-dd'T'HH:mm:ss.SSS'Z', UTC} | %-20.20thread | %5level | %-25.25logger{25} | %m%n</pattern>
    </encoder>
  </appender>

  <appender name="LOGBUFFER" class="io.deephaven.logback.LogBufferAppender">
    <encoder>
      <pattern>%-20.20thread | %-25.25logger{25} | %m</pattern>
    </encoder>
  </appender>

  <root level="info">
    <appender-ref ref="STDOUT" />
    <appender-ref ref="LOGBUFFER" />
  </root>
</configuration>

The LevelChangePropagator listener copies Logback's log levels to Java's built-in logging. Libraries that log through it, such as gRPC, then skip messages that Logback would discard instead of building them first.

Run the application

Start the application with Gradle:

./gradlew run

Or build a distribution with a launch script, and run that:

./gradlew installDist
./build/install/my-deephaven-app/bin/my-deephaven-app

The generated launch script includes the JVM arguments from applicationDefaultJvmArgs and adds anything in the JAVA_OPTS environment variable.

When the server is ready, the log shows Server started on port 10000. By default, the server uses pre-shared key authentication with a generated key, and logs a URL that includes the key. The default key comes from java.util.Random, which isn't cryptographically secure, so set your own strong key with -Dauthentication.psk=<key> (see Configure the server) or choose another authentication handler before you expose the server beyond your own machine:

Connect automatically to Web UI with http://localhost:10000/?psk=<key>

Warning

The pre-shared key is a superuser credential: anyone who presents it is authenticated as a superuser. The log line above, including the key and the clickable URL, goes to both stdout and the log buffer that the web UI's Log panel reads from (see Configure logging). Restrict access to these logs, whether on disk, in a collection system, or in the running server's own UI, to avoid leaking superuser access.

Open that URL to use the web UI. The Groovy console works as it does in any Deephaven server, and the Panels menu lists the ticking and staticTable fields from MyApplication.

Configure the server

A custom application reads the same configuration as the production application:

Configuration properties, set as JVM system properties or in a configuration file. For example, -Dhttp.port=8080 changes the port, and -Dauthentication.psk=<key> sets the pre-shared key. See Configuration properties for more.
The bootstrap settings for the application name and the data, cache, and config directories, described in Configure the production application. The example build sets deephaven.application to my-deephaven-app. Without it, the application name defaults to deephaven, so your application shares its data, cache, and config directories, including any deephaven.prop file, with other Deephaven servers on the same machine.
The authentication handlers, chosen with the AuthHandlers property. See Available authentication handlers for the built-in options, and the authentication guides for how to configure each one.

For example, to set the key and port when using the launch script:

JAVA_OPTS="-Dauthentication.psk=YOUR_PASSWORD_HERE -Dhttp.port=8080" ./build/install/my-deephaven-app/bin/my-deephaven-app