View etcd history

Under normal operation, you should avoid accessing data (such as property files or the routing file) directly using etcd commands. However, if a file has been deleted or overwritten with an invalid file and no backup is available, it may be possible to retrieve previous versions.

Warning

etcd history is not meant for recovery. Previous file versions may not be accessible due to automatic compaction. These steps should only be taken as a last resort and cannot replace a proper backup strategy.

Files in etcd are stored using a pseudo-directory structure, and metadata is written each time a file is updated. The wrtmr file shows the revision number of the previous update. The following command searches for this file and displays the revision number for each file and directory level in etcd.

sudo -u irisadmin DH_ETCD_DIR=/etc/sysconfig/deephaven/etcd/client/root /usr/illumon/latest/bin/etcdctl.sh get --prefix /main | grep -A1 __meta\.*wrtmr

Search the output for the desired file, such as routing-file or iris-environment. Note that some characters may be changed to meet key path naming requirements. You should see an /updated/wrtmr with a number. For example:

/main/config/props/files/iris-environment-prop/__meta/updated/wrtmr
497814

Or, if the file is deleted, you might need to go up one or more levels:

/main/config/__meta/updated/wrtmr
1554541

To find the file, subtract one from the wrtmr value and retrieve the file from etcd before that revision. The following examples retrieve the iris-environment property file and the routing file from the above values:

sudo -u irisadmin  DH_ETCD_DIR=/etc/sysconfig/deephaven/etcd/client/root /usr/illumon/latest/bin/etcdctl.sh get --prefix /main/config/props/files/iris-environment-prop/file --rev=497813

and

sudo -u irisadmin  DH_ETCD_DIR=/etc/sysconfig/deephaven/etcd/client/root /usr/illumon/latest/bin/etcdctl.sh get --prefix /main/config/routing-file/file --rev=1554540

If the revision number is too old, you may receive an error stating required revision has been compacted. In this case, you must use a newer revision, and the previous file contents might not be recoverable.

Review the output to confirm it contains the expected data. Then, paste it into a file and follow the standard dhconfig instructions to import it.