---
title: Persistent Query Controller etcd access
---

If a malformed Persistent Query (PQ) XML is stored in etcd, the controller may fail to start. This guide explains how to access the controller's etcd storage directly to remove the problematic PQ. This procedure should only be used in exceptional circumstances.

> [!WARNING]
> These instructions should only be used when the controller fails to start due to a problematic PQ. If this occurs, please contact Deephaven support.

Every PQ is uniquely identified by a serial number that is displayed in the exception message. For example, the serial `1660929719014000000` converts to `170ccec78ab80580` in hexadecimal. When interacting with etcd, always use the hexadecimal format with lowercase letters. Update the examples with the converted hexadecimal serial.

All interactions with etcd will use the Deephaven administration system account, typically `irisadmin`. These examples assume that etcd is configured with its default Deephaven settings, so the appropriate keys are accessible to `irisadmin` in the directory `/etc/sysconfig/illumon.d/etcd/client/controller`. Running the commands on the server where the controller operates will ensure this.

To see a list of PQ serials in hexadecimal, use the following command:

```
sudo -u irisadmin \
DH_ETCD_USER=controller /usr/illumon/latest/bin/etcdctl.sh get \
 --prefix /main/data/persistent-query-v2/ \
 --keys-only
```

To display the details of a PQ, use the following command. This will print an XML view of the PQ:

```
sudo -u irisadmin \
DH_ETCD_USER=controller /usr/illumon/latest/bin/etcdctl.sh get \
 /main/data/persistent-query-v2/configuration/170ccec78ab80580 \
 --print-value-only
```

To delete a PQ, use the following command. This should only be done in emergencies and when the PQ controller is stopped. If there are multiple controllers, ensure all of them are stopped before proceeding.

```
sudo -u irisadmin \
DH_ETCD_USER=controller /usr/illumon/latest/bin/etcdctl.sh del \
 --prefix /main/data/persistent-query-v2/configuration/170ccec78ab80580
```

## Related documentation

- [etcd](../core-components/etcd.md)