Configuration properties backup and restoration

This guide demonstrates how to import and export Deephaven configuration, either as a complete suite or as individual components.

The sections below demonstrates how to export and import each restorable configuration type:

• ACLs
• Persistent Queries
• Schema files
• Routing files
• Property files

Backups may be done with the Deephaven backup script or exported individually, as demonstrated below.

Configuration backup

The data in etcd can be exported as separate files and imported into the new cluster with the dhconfig tool. This allows the transfer of specific sets of properties instead of the entire dataset.

Individual configuration backup

Backups for the complete suite or individual components of configurations may be done in a single step with the Deephaven backup script. Alternatively, individual configuration types can be exported with the following commands (shown with their backup-script equivalent):

# Each of the following commands is run as the "irisadmin" user. This may be different on your
# system, depending on installation properties. In all cases, the command should be run from
# a directory where this user has write permissions OR the "filename" should be a full path
# to a directory where the user has write permissions. The same goes for "pre_existing_path";
# the user must have write access to the directory.

# ACL export ("-A" or "--ACLs" from backup script) with:
sudo -u irisadmin /usr/illumon/latest/bin/dhconfig acl export --file [filename]

# Persistent Queries backup ("-p" or "--persistentQueries" from backup script) with:
sudo -u irisadmin /usr/illumon/latest/bin/dhconfig pq export --file [filename]

# Schemas backup ("-s" or "--schema" from backup script) with:
sudo -u irisadmin /usr/illumon/latest/bin/dhconfig schema export --directory [pre_existing_path]

# Primary routing backup (included in "-r" or "--routing" from backup script) with:
sudo -u irisadmin /usr/illumon/latest/bin/dhconfig routing export --file [filename]

# DIS routing backup (included in "-r" or "--routing" from backup script) with:
sudo -u irisadmin /usr/illumon/latest/bin/dhconfig dis export --directory [pre_existing_path]

# Property files backup ("-P" or "--propertyFiles" from backup script) with:
sudo -u irisadmin /usr/illumon/latest/bin/dhconfig properties export --directory [pre_existing_path]

There are several options for exporting configurations. For more details on additional options, see the full dhconfig tool documentation.

Configuration restore

The following section demonstrates how to restore configuration, whether backed up with the Deephaven backup script or individually exported with the dhconfig tool.

ACL DB restore

The dhconfig tool can be used to import ACL data to any Deephaven deployment. See dhconfig acl for more details on additional options.

Specifically, to import ACLs with dhconfig acl, use the following command:

# ALC restore with:
sudo -u irisadmin /usr/illumon/latest/bin/dhconfig acl import --file [filename]

ACL files for export and import can be .xml or .json. The options --overwrite or --replace-all are often used with dhconfig acl import to direct the system on how to handle the data being imported with respect to any existing data. These options are covered in more detail in the Editing ACLs section of the Permissions guide.

Persistent Queries restore

Persistent Query configurations can be imported with the following command:

# Persistent Queries restore with:
sudo -u irisadmin /usr/illumon/latest/bin/dhconfig pq import --retain-serial --file [filename]

There are a number of options for importing Persistent Queries. See the full dhconfig pq documentation for additional options. In particular:

• If you're restoring to a different Deephaven installation, the --server-override option may be useful.
• If you want to include temporary Persistent Queries, add the --include-temporary parameter. Temporary Persistent Queries will be re-run when they're added. Since temporary Persistent Queries are typically used for one-time tasks, they are not imported by default.
• If you want to re-import the helper queries, then include the --include-non-displayable parameter.

Schema file restore

Schema files are placed into a .tar file by the Deephaven backup script, so if the backup was created with that tool, the first step is to extract the schemas on the target system. If the configuration was exported directly with dhconfig schema export --directory, the following step can be skipped. To extract schemas from the .tar file:

# Create a unique temporary directory and extract the schema tar file into it
TEMP_SCHEMA_DIR=$(sudo -u irisadmin mktemp -d -t schemas.XXXXXX)
sudo -u irisadmin tar -xvf [schemas_file].tar  -C ${TEMP_SCHEMA_DIR}

Then, you can import the schema files from the directory.

To import all the schemas in a directory:

# Schemas restore with:
sudo -u irisadmin /usr/illumon/latest/bin/dhconfig schema import --directory [pre_existing_path]

See the full dhconfig schema documentation for details on schema import options. In particular:

• The --namespace option can restrict the imports to specific namespaces.
• The --force option overwrites any existing schemas on the target system with the versions from this directory.

Routing files restore

Two types of routing files can be restored: routing files exported with the dhconfig routing export command and routing files exported with the dhconfig dis export command.

To import:

# Primary routing restore with:
sudo -u irisadmin /usr/illumon/latest/bin/dhconfig routing import --file [filename].yml

# DIS routing restore with:
sudo -u irisadmin /usr/illumon/latest/bin/dhconfig dis import --file [dis_routing_file].yml

The dhconfig dis import will fail if the routing configurations already exist on the system. Add --force to overwrite existing configurations. To import a single configuration from the file, extract the appropriate section into a new file. See the full dhconfig routing and dhconfig dis documentation for details on the routing import options.

Property files restore

The Deephaven backup script places property files into a .tar file, so if the backup was created with that tool, the first step is to extract the schemas on the target system. If the configuration was exported directly with dhconfig properties export —-directory, the following step can be skipped.

To extract property files from the .tar file:

# Create a unique temporary directory and extract the properties tar file into it
TEMP_PROP_DIR=$(sudo -u irisadmin mktemp -d -t props.XXXXXX)
sudo -u irisadmin tar -xvf [props_file].tar -C ${TEMP_PROP_DIR}

You can import individual property files with the command:

# For each desired property file, restore with:
sudo -u irisadmin /usr/illumon/latest/bin/dhconfig properties import --file [filename]

Related documentation

• Backup, Restore, and Migration overview
• etcd and MySQL backup and restore
• File system backup and restore
• Table migration
• Workspace data backup and restore