dhconfig dis
The dis (or data_import_server) configuration data type of the dhconfig tool is used to work with Data Import Server (DIS) configurations. The main data routing configuration file includes a section called dataImportServers, which defines DIS (Data Import Server) configurations. For more details, see the documentation for dhconfig routing and the Data routing YAML format. This command manages DIS configurations directly, separate from the main configuration. This method is much more convenient than editing the data routing configuration for most additional DIS configurations. A DIS may be configured either in the main routing configuration, or separately using this command.
Note
Unless you use the --etcd option, the authenticated user must be in a group authorized to make data routing service changes. The superusers group (iris-superusers by default) always has write permission. Additional groups can be given permission with the following property, which must be visible to the configuration server:
Usage:
As with other dhconfig configuration data types, the --help argument provides detailed information on the available actions and arguments, as well as usage examples. Use --help whenever you need more information about the available options. For example, if you want information about the import action, you can run:
Which prints the following:
A description of all the subcommand's options and some usage examples follows.
Due to the number of available options and combinations, and the robust in-tool documentation via --help, an exhaustive list of all possible commands is not provided here. Instead, we will elaborate on some of the more complex arguments below, and then give some usage examples. We recommend that you use the --help argument as needed for more information.
Note
Many Deephaven processes respond dynamically to data routing configuration changes.
Note
DIS (Data Import Server) names that are configured separately — outside of the main data routing configuration file — must follow stricter naming rules. For best compatibility, use only letters, numbers, underscores, and dashes in DIS names, and avoid special characters.
dis subcommands
The following subcommands are supported for dhconfig dis:
Note
Bypassing configuration server with --etcd:
When you use the --etcd option, dhconfig connects directly to the etcd database, skipping the Configuration Server. This should only be done by administrators with the necessary permissions and is typically used for troubleshooting or recovery when the Configuration Server is unavailable.
export
Prints or exports the data import server configurations in the system.
| Argument | Description |
|---|---|
--name <arg> | Export only the named data import server configurations. It may be specified multiple times. Trailing arguments are treated as --name arguments. |
--directory | specify the directory for writing files. If specified, data import server configurations are written to individual files in this directory. Cannot be combined with --file. |
--file | export the specified configurations to this filename. Optional, defaults to stdout. |
--ignore-errors | Export the configuration(s) even when there are parsing errors in the stored configurations. Only valid with --etcd. |
--all | Include core data import server configurations in the export. WARNING: This creates a configuration that cannot be imported without editing the main data routing file. |
Usage examples:
Print separately configured data import servers:
Export all separately configured data import servers to a single file:
Export all separately configured data import servers to individual files (the directory must exist):
Export a data import server configuration that is defined in the routing file. All anchors are resolved. To import the result, first remove that data import server from the main routing file, because import fails for a name that is already defined there:
import
Loads one or more DIS configurations from a file or files. See the configuration generated by the export command to see the format. If the file contains multiple configurations, they must be separated by --- lines, for example:
--- is a standard YAML convention for delimiting multiple documents within a single file.
| Argument | Description |
|---|---|
--force | Import the data import servers even if configurations with the same names already exist. |
--clobber | Delete all existing data import server configurations, and replace them with the set given in this command. |
--ignore-errors | Without --clobber, import even when a DIS filter references InternalPartition without handling TableLocationKey.NULL_PARTITION; parse errors still fail. With --clobber, also set the configuration even if the existing or new configuration has parsing errors. Use with extreme caution. |
--routing-file <arg> | With --clobber, include the given routing file along with the data import server configurations. |
--file | Input file(s) to import. It may be included multiple times in most contexts. Trailing arguments are treated as --file arguments. |
Note
A whereLocationKey filter that references InternalPartition must also account for the aggregate-key placeholder, or DIS truncate/delete commands will be rejected. Include a clause such as InternalPartition == TableLocationKey.NULL_PARTITION || ... in the filter; see the whereLocationKey section of the YAML reference for details. import and edit will fail if this is not handled; pass --ignore-errors to proceed with the filter as written.
Usage examples:
Note
Unless you use --etcd, import actions require authentication, with sudo, --key, or --user.
Import data import server configuration(s) from /tmp/import_servers.yml:
Import data import server configuration from files in /tmp/import_servers:
Import data routing configuration from /tmp/import_servers.yml, bypassing configuration server (requires sudo):
Replace the set of data import server configurations with those in file /tmp/import_servers.yml (requires sudo):
This replaces the separately managed data import server configurations. If the stored configuration is invalid, add --ignore-errors, or supply a valid --routing-file as in the next example.
Replace the entire data routing configuration with the data routing file /tmp/routing.yml and set of data import server configurations from the file in /tmp/import_servers.yml (requires sudo):
To repair a damaged configuration, use --etcd, which connects directly to etcd and works without the Configuration Server. Adding --ignore-errors to the previous command also sets the new configuration when validation reports errors:
Warning
Use with extreme caution because this can create an invalid configuration.
edit
Note
Unless you use --etcd, edit actions require authentication, with sudo, --key, or --user.
Opens separately configured data import server configurations in an editor. DIS configurations defined in the main data routing file are not included; use dhconfig routing edit for those. After you save and exit, dhconfig asks whether to import the changed file, then validates it as import does. If validation fails, you can edit again, show the full exception, retry ignoring the InternalPartition filter check, or abort. See Editing configuration for more information.
| Argument | Description |
|---|---|
--name <arg> | Edit only the named data import server configurations. It may be specified multiple times. Trailing arguments are treated as --name arguments. |
-e,--editor <arg> | The editor to use. Optional; defaults to the EDITOR environment variable, or vi if it is not set. |
--ignore-errors | Save even when a DIS filter references InternalPartition without handling TableLocationKey.NULL_PARTITION. When you edit all configurations (no --name), also save even if validation reports errors. Use with caution. |
Usage examples:
Edit data import server configurations extra_dis and extra_dis2:
Edit all separately configured data import servers. Any configuration you remove in the editor is deleted:
add
Defines and adds a new DIS configuration.
| Argument | Description |
|---|---|
--name <arg> | The name of a data import server configuration to add. |
--claim <arg> | Claim the specified namespace or namespace.tableName for this import server. It may be specified multiple times. |
--force | Add the data import server even if a configuration with this name already exists. |
Usage examples:
Note
Unless you use --etcd, add actions require authentication with sudo, --key, or --user.
Define and import a data import server configuration claiming several namespaces:
Define and import a data import server configuration claiming several tables:
list
Lists the data import server configurations in the system.
| Argument | Description |
|---|---|
--all | List all data import server configurations, including core configurations. |
--long | Include claims made by all listed data import servers. |
Usage examples:
List separately configured data import servers in routing configuration:
List all configured data import servers in routing configuration (including those defined in the main routing file):
delete
Removes one or more data import server configurations from the system.
| Argument | Description |
|---|---|
--name <arg> | The name of a data import server configuration to delete. It may be specified multiple times. Trailing arguments are treated as --name arguments. |
-l,--lenient | Don't require the named data import server configurations to exist. |
--force | Really delete the indicated configurations. Required for safety. |
--ignore-errors | Delete the configuration(s) even if the existing or new configuration has errors. Use with extreme caution. |
Usage examples:
Note
Unless you use --etcd, delete actions require authentication, with sudo, --key, or --user.
Without --force, delete reports what it would delete, and flags names that do not exist or that refer to core configurations defined in the main routing file, which cannot be deleted with this command.
Delete data import servers from routing configuration:
validate
Checks a file or files for errors without importing them.
| Argument | Description |
|---|---|
--file | Input file(s) to validate. It may be included multiple times in most contexts. Trailing arguments are treated as --file arguments. |
--routing-file <arg> | Validate including the given routing file along with the data import server configurations. |
Usage examples:
Verify that the files parse, and will not cause errors when added to the routing configuration:
show-claims
Shows existing data import server claims.
| Argument | Description |
|---|---|
--namespace <arg> ... | Include only the given namespace(s). To specify multiple namespaces, list them as separate arguments (e.g., --namespace NS1 NS2). |
--table-name | Include only the given table (requires exactly one --namespace). |
Usage examples:
Show existing data import server claims:
Show claims for the namespaces AAA, BBB, and CCC:
Show claims for the table AAA.BBB: