---
id: runbook-acl-write-server
title: ACL Write Server runbook
---

The Access Control List (ACL) Write Server is a Deephaven infrastructure service that provides the API for modifying ACL data including users, groups, and permissions. It is the only service that can write to the ACL database and requires membership in the `iris-superusers` group to access.

## Impact of ACL Write Server failure

| Level            | Impact                                                                 |
| :--------------- | :--------------------------------------------------------------------- |
| Sev 2 - Moderate | Administrators will not be able to update user permissions and groups. |

> [!NOTE]
> ACL failures do not prevent users from working. Existing permissions remain in effect, but administrators cannot modify user access, create new users, or change group memberships until the ACL Write Server is restored.

## ACL Write Server purpose

The ACL Write Server provides the administrative interface for managing:

- **Users** — Create, modify, and delete user accounts.
- **Groups** — Create, modify, and delete groups and manage memberships.
- **Permissions** — Grant and revoke table, query, and system permissions.
- **DSA keys** — Manage public key authentication for programmatic access.
- **Password management** — Reset passwords, enforce password policies.

## ACL Write Server dependencies

The ACL Write Server requires:

1. **etcd cluster** — Must be running and accessible if using etcd for ACL storage.
2. **MySQL/MariaDB** — Must be running if using MySQL for ACL storage.
3. **etcd client configuration files** — Located at `/etc/sysconfig/deephaven/etcd/client`.
4. **Authentication Server** — Used to validate requesting user has superuser permissions.

## Checking ACL Write Server status

Check process is running with monit:

```bash
dh_monit status db_acl_write_server
```

Expected output should show status `Running`.

## Viewing ACL Write Server logs

View application log:

```bash
cat /var/log/deephaven/acl_write_server/DbAclWriteServer.log.current
```

Tail the log to follow in real-time:

```bash
tail -f /var/log/deephaven/acl_write_server/DbAclWriteServer.log.current
```

List historical log files:

```bash
ls -ltr /var/log/deephaven/acl_write_server/db_acl_write_server.log.????-??-??
```

View process stdout/stderr logs:

```bash
cat /var/log/deephaven/acl_write_server/db_acl_write_server.log.$(date +%Y-%m-%d)
```

## Restart procedure

Restart the ACL Write Server:

```bash
dh_monit restart db_acl_write_server
```

**Impact:** Restarting the ACL Write Server temporarily prevents ACL modifications but does not affect existing user sessions or permissions.

Verify the restart was successful:

```bash
dh_monit status db_acl_write_server
```

Monitor the log during startup:

```bash
tail -f /var/log/deephaven/acl_write_server/DbAclWriteServer.log.current
```

**Expected startup messages:**

- Successful connection to ACL backend (etcd or MySQL).
- Server listening on configured port.
- No authentication backend errors.

## Checking dependencies

### Check etcd status

If using etcd for ACLs:

```bash
sudo -u irisadmin /usr/illumon/latest/bin/etcdctl.sh endpoint status --write-out table
```

### Check MySQL status

If using MySQL for ACLs:

```bash
sudo systemctl status mariadb
```

Test MySQL connectivity and query ACL tables:

```bash
sudo mysql -D dbacl_iris -e "SELECT COUNT(*) FROM users"
sudo mysql -D dbacl_iris -e "SELECT COUNT(*) FROM groups"
sudo mysql -D dbacl_iris -e "SELECT COUNT(*) FROM tableacls"
```

## Managing ACLs with dhconfig

The primary interface for ACL management is the `dhconfig acls` command.

### List users

```bash
sudo /usr/illumon/latest/bin/dhconfig acls users list
```

### Create a user

```bash
sudo /usr/illumon/latest/bin/dhconfig acls users add --name newuser --hashed-password '<hashed-password>'
```

### Modify user permissions

System-level permissions (such as `superuser`) are managed through ACL import/export. See the [dhconfig acls command reference](../configuration/dhconfig/acls.md) for the `import` and `export` subcommands with `--type systemuser`.

### List groups

```bash
sudo /usr/illumon/latest/bin/dhconfig acls groups list
```

### Create a group

Groups are created implicitly when members are added, or via ACL import. There is no standalone group-creation command. To create a group and add a member in one step:

```bash
sudo /usr/illumon/latest/bin/dhconfig acls groups add-member --name someuser --group analysts
```

### Add user to group

```bash
sudo /usr/illumon/latest/bin/dhconfig acls groups add-member --name someuser --group analysts
```

### Grant table permissions

Table ACLs are managed through ACL import/export. See the [dhconfig acls command reference](../configuration/dhconfig/acls.md) for the `import` and `export` subcommands with `--type tableacls`.

### Manage public keys

```bash
# Import a public key for programmatic access
sudo /usr/illumon/latest/bin/dhconfig acls publickeys import --file /path/to/public_key.pem
```

See [dhconfig acls command reference](../configuration/dhconfig/acls.md) for complete documentation.

## ACL storage backends

The ACL Write Server supports two storage backends:

### etcd backend (recommended)

**Advantages:**

- Integrated with other Deephaven configuration.
- Better consistency with cluster configuration.
- Simpler backup/restore with etcd snapshots.
- No separate database to manage.

**Configuration:**

```properties
acls.dataSource=etcd
```

### MySQL backend (Legacy)

**Advantages:**

- May be required for Legacy migrations.
- Familiar SQL interface for direct queries.
- Existing MySQL infrastructure can be reused.

**Configuration:**

```properties
acls.dataSource=mysql
db.url=jdbc:mariadb://localhost:3306/dbacl_iris
db.user=iris
db.password=<encrypted-password>
```

## Backup and recovery

### Backup ACL data

**For etcd-based ACLs:**

ACLs are backed up as part of etcd snapshots:

```bash
sudo -u irisadmin /usr/illumon/latest/bin/etcdctl.sh snapshot save /backup/etcd-acls-$(date +%Y%m%d).db
```

**For MySQL-based ACLs:**

Create MySQL backup:

```bash
sudo mysqldump dbacl_iris > /backup/acls-$(date +%Y%m%d).sql
```

### Restore ACL data

**For etcd:** Restore from etcd snapshot (see [etcd runbook](runbook-etcd.md))

**For MySQL:**

```bash
sudo mysql dbacl_iris < /backup/acls-20250101.sql
```

## Migration from MySQL to etcd

To migrate ACLs from MySQL to etcd:

See [Migrating ACLs to etcd](../installation/basic-install.md#appendix-g-migrating-acls-to-etcd) for detailed procedure.

## Configuration files and locations

**monit configuration:** `/etc/sysconfig/illumon.d/monit/db_acl_write_server.conf`

**Property files:**

- `/etc/sysconfig/illumon.d/resources/iris-common.prop`
- `/etc/sysconfig/illumon.d/resources/db_acl_write_server.prop`

**MySQL database:** `dbacl_iris` (if using MySQL backend)

**MySQL configuration:** `/etc/my.cnf`

**Log directory:** `/var/log/deephaven/acl_write_server/`

## Related documentation

- [System processes overview](../architecture/architecture-overview.md)
- [Permissions overview](../permissions/permissions-overview.md)
- [ACL Editor](../permissions/web-acl-editor.md)
- [Managing permissions](../security/hardening-technical-controls.md)
- [dhconfig acls command reference](../configuration/dhconfig/acls.md)
- [Authentication Server runbook](runbook-authentication-server.md)
- [etcd runbook](runbook-etcd.md)