Installing Envoy

This guide provides instructions for installing, upgrading, and restarting Envoy as Deephaven's front proxy in all three supported Deephaven installation types. Instructions will be given for Kubernetes and Podman first, as they have a much simpler operational model.

Install Envoy on Kubernetes

Envoy is always used with Kubernetes.

All you need to do is supply a signing certificate and configure the hostname and port where Deephaven clients can reach the exposed Envoy Kubernetes service.

Install Envoy on Podman

Podman installation supports using Envoy by passing the --envoy|-e flag to start_command.sh when creating your pod. If you build custom images, you must provide the Envoy binary to build_images.sh.

By default, Envoy on Podman will use the same TLS certificate as all other processes on the dh-infra pod. You may optionally place a "signing certificate and private key" PEM formatted envoy.pem bundle in the --tls-volume directory passed to start_command.sh.

Install Envoy on native installations

Deephaven installations can now automatically install, update, and restart the Envoy reverse proxy.

Tip

If you are upgrading a cluster which has a pre-existing, manually managed Envoy process, you must manually Uninstall your previous Envoy installation before you can opt in to the Deephaven-managed Envoy installation process.

Manually-managed Envoy

By default, Envoy is neither installed nor restarted when the native installer runs, due to DH_ENVOY_INSTALL_MODE having a default value of None.

This policy ensures smooth upgrades for customers who have already set up Envoy on their own.

In the manually managed scenario, the Deephaven native installer configures the cluster to speak to Envoy for you, and generates the appropriate envoy.yaml for Envoy to use, but it does not install or restart Envoy.

Deephaven-managed Envoy

When DH_CONFIGURE_ENVOY=true, the installer can manage Envoy using the DH_ENVOY_INSTALL_MODE options:

DOCKER / PODMAN: pulls and runs the Envoy image and mounts the generated YAML and the Envoy endpoint certificate PEM. It creates a systemd service unit called deephaven_envoy to simplify process restarts and integrate with journalctl logging.
NATIVE: installs Envoy as a native systemd user service run by DH_ADMIN_USER.
NONE: does not install, configure, or restart Envoy.

Select a single node within the Deephaven cluster to run Envoy and ensure it has network access to the Deephaven services it will front.

Note

Before you begin, check the Deephaven version support matrix to identify the Envoy version compatible with your Deephaven release.

Envoy installation methods

Tip

This section is only relevant to native installation methods. Podman and Kubernetes are covered at the start of the document.

While many Envoy installation methods exist, Deephaven recommends using Deephaven-managed PODMAN if it is available, as Podman quadlets have the most elegant systemd integration. On Ubuntu installations, DOCKER has good integration with systemd, while NATIVE is harder to use, but works without a container or root permission.

The following sections provide instructions for common installation methods. For comprehensive details or alternative methods, refer to the official Envoy installation documentation.

Example Podman quadlet

When DH_ENVOY_INSTALL_MODE=PODMAN, Deephaven will generate a Podman quadlet file.

The sample Podman quadlet file below is configured with default Envoy port and Envoy admin port, run as the default Admin service account, using default locations for envoy.yaml and Envoy certificates.

/etc/containers/systemd/deephaven_envoy.container

[Unit]
Description=Envoy proxy server (deephaven_envoy)
Wants=network-online.target
After=network-online.target

[Container]
ContainerName=deephaven_envoy
Image=docker.io/envoyproxy/envoy:v1.36.4

PublishPort=8000:8000
PublishPort=8001:8001

Volume=/etc/sysconfig/illumon.d/resources/envoy3.yaml:/config.yaml:rw
Volume=/etc/deephaven/certs/envoy.pem:/envoy.pem:ro

User=9002
Exec=-c /config.yaml

[Service]
Restart=always
TimeoutStopSec=70

[Install]
WantedBy=default.target

Be sure to check that you are using the correct Envoy version, that your necessary ports are exposed, and that your envoy.yaml and certificates are correctly mounted.

After creating the quadlet file, Deephaven will update systemd:

# reload systemd to generate deephaven_envoy.container
systemctl daemon-reload
# no need to enable the service as it is generated
systemctl start deephaven_envoy.service

The envoy3.yaml should specify the certificate location it expects within the container; in this example, /envoy.pem.

Example Docker service

When DH_ENVOY_INSTALL_MODE=DOCKER, Deephaven will generate a systemd service file which attaches directly to the invocation of docker run --rm. This allows systemd to immediately detect and respond to process failures.

The sample systemd service unit below uses the default Envoy port and Envoy admin port, run as the default Admin service account, using default locations for envoy.yaml and Envoy certificates.

/etc/systemd/system/deephaven_envoy.service

[Unit]
Description=Envoy proxy server (deephaven_envoy)
Wants=network-online.target
After=network-online.target

[Service]
Restart=always
RestartSec=5
TimeoutStopSec=70

# Ensure we don't fail due to a stale container name
ExecStartPre=/usr/bin/docker rm -f deephaven_envoy

ExecStart=/usr/bin/docker run \
--name deephaven_envoy \
--rm \
-p 8000:8000 \
-p 8001:8001 \
-v /etc/sysconfig/illumon.d/resources/envoy3.yaml:/config.yaml:rw \
-v /etc/deephaven/certs/envoy.pem:/envoy.pem:ro \
-u 9002 \
docker.io/envoyproxy/envoy:v1.36.4 \
-c /config.yaml

ExecStop=/usr/bin/docker stop --timeout 10 deephaven_envoy

[Install]
WantedBy=multi-user.target

Be sure to check that you are using the correct Envoy version, your necessary ports are exposed, and that your envoy.yaml and certificates are correctly mounted.

After creating the service file, Deephaven will reload systemd:

# Tell systemd to process our new service
systemctl daemon-reload
# Enable and start the service.
systemctl enable --now deephaven_envoy.service

The envoy3.yaml should specify the certificate location it expects within the container; in this example, /envoy.pem.

Example native Envoy installation

When using DH_ENVOY_INSTALL_MODE=NATIVE, Deephaven will create a systemd service unit that does not require any root privileges and starts no containers. Deephaven directly downloads the Envoy binary from Envoy's GitHub releases page, and runs it as the default Administrative service account, irisadmin.

The sample systemd user unit service file below uses the default Admin service account username of irisadmin, and requires that you have downloaded an Envoy binary file to ~irisadmin/bin/envoy-1.36.4-linux-x86_64.

~irisadmin/.config/systemd/user/deephaven_envoy.service

[Unit]
Description=Envoy proxy server (deephaven_envoy) - native binary
Wants=network-online.target
After=network-online.target

[Service]
Restart=always
RestartSec=5
TimeoutStopSec=70

# Run Envoy using the installer-managed config + cert locations.
ExecStart=%h/bin/envoy-1.36.4-linux-x86_64 -c /etc/sysconfig/illumon.d/resources/envoy3.yaml
WorkingDirectory=%h

[Install]
WantedBy=default.target

When setting up a native process in systemd, you do not need to worry about port mapping or volume mounting, but you must be sure that your envoy.yaml uses the correct ports and paths to your TLS signing certificate.

After creating the user service file, Deephaven will enable and start the service using systemctl:

sudo -u irisadmin bash -c '
  # This variable must be set to use systemd user units
  export XDG_RUNTIME_DIR=${XDG_RUNTIME_DIR:-/run/user/$(id -u)}
  # Tell systemd to process our service file
  systemctl --user daemon-reload
  # Enable and start the unit
  systemctl --user enable --now deephaven_envoy.service
'

In order to use systemd user units, special care must be taken to set the XDG_RUNTIME_DIR, especially when sudo is used:

# systemd user unit operations require extra care:
sudo -u irisadmin XDG_RUNTIME_DIR=/run/user/$(id -u irisadmin) \
      systemctl --user status deephaven_envoy
# Journalctl does not require --user flags or sudo -u
journalctl -eu deephaven_envoy

While this installation method is a little harder to use once installed, it requires no special permissions.

Manual Envoy installation

There are many documented installation methods that are maintained by the Envoy team.

You may choose any of those options, and should explicitly set DH_ENVOY_INSTALL_MODE=NONE; even though it is the default installation mode, it is best to be explicit about your configuration.

However you choose to install Envoy, ensure you use the Deephaven-tested Envoy version listed in the product support matrix.

For example, on Ubuntu systems, if you follow Envoy's installation instructions, make sure to specify the Envoy version when installing with apt-get:

sudo apt-get update
sudo apt-get install envoy=1.36.4
envoy --version

Manual Envoy upgrade

In order to manually upgrade Envoy, you should first Uninstall Envoy and then Reinstall Envoy.

Uninstall manual Envoy

If you are upgrading a system that already has Manually-managed Envoy, before you turn on Deephaven managed Envoy, you should first remove or at least disable your previous Envoy installation.

If you installed Envoy using Deephaven documentation from previous versions, you may need to remove both a running container, and a systemd service unit.

The first step to uninstalling Envoy is to determine the installation method used.

Once you have determined how Envoy is installed, you can decide how to uninstall it.

For containerized installation:

docker stop deephaven_envoy
docker rm deephaven_envoy
#or
podman stop deephaven_envoy
podman rm deephaven_envoy

For native installation, or for installations which have both a containerized component and a systemd unit, you may also need to remove a systemd unit:

systemctl stop deephaven_envoy
systemctl disable deephaven_envoy

Once uninstallation is complete, verify that Envoy is no longer running:

ps -ef | grep -v grep | grep envoy

Restarting Envoy

How you restart Envoy depends on the ownership model:

For installer-managed DOCKER and PODMAN modes, restart the deephaven_envoy systemd unit:
```
sudo systemctl restart deephaven_envoy
```

For installer-managed NATIVE mode, restart the user service:

sudo -u irisadmin XDG_RUNTIME_DIR=/run/user/$(id -u irisadmin) \
   systemctl --user restart deephaven_envoy

For DH_ENVOY_INSTALL_MODE=NONE, use whatever restart method your manual deployment requires.

Caution

A full installation or upgrade of your cluster will overwrite configuration and certificate files, and require all processes including Envoy to be restarted. Using dh_keygen.sh to rotate certificates will update certificates but does not restart Envoy or Deephaven services; you will need to manually restart processes yourself.

When Deephaven-managed Envoy is used, full installations will restart Envoy automatically.

If you have only changed the envoy.yaml, you can hot reload the Envoy configuration to load the new configuration and drain existing connections without restarting.