---
id: runbook-log-aggregator-service
title: Log Aggregator Service runbook
---

The [Log Aggregator Service (LAS)](../core-components/log-aggregator-service.md) is a critical Deephaven data management service that combines binary log entries from multiple processes and writes them into combined binary log files. It prevents write contention by serializing writes to each output stream, ensuring only one writer per log location at a time.

## Impact of Log Aggregator Service failure

| Level            | Impact                                                                                                                                                  |
| :--------------- | :------------------------------------------------------------------------------------------------------------------------------------------------------ |
| Sev 1 - Critical | Any process configured to use the LAS will fail to write logs to the database. This will cause failure of these processes, including the query workers. |

> [!CAUTION]
> Workers are typically configured to require successful LAS connection before starting. If LAS is down, new workers cannot start.

## Log Aggregator Service dependencies

The LAS requires:

1. **Configuration Server** — Must be running to access routing and schema configuration.
2. **Authentication Server** — Must be running for token validation.
3. **Filesystem access** — Must have read/write access to binary log storage paths.
4. **etcd cluster** — Must be accessible (via Configuration Server).

The LAS does not depend on [Remote Query Dispatcher](runbook-remote-query-dispatcher.md), allowing it to start independently.

## Checking Log Aggregator Service status

Check process is running with monit:

```bash
dh_monit status log_aggregator_service
```

Expected output should show status `Running`.

## Viewing Log Aggregator Service logs

View application log:

```bash
cat /var/log/deephaven/las/LogAggregatorService.log.current
```

Tail the log to follow in real-time:

```bash
tail -f /var/log/deephaven/las/LogAggregatorService.log.current
```

List historical log files:

```bash
ls -ltr /var/log/deephaven/las/log_aggregator_service.log.????-??-??
```

View process stdout/stderr logs:

```bash
cat /var/log/deephaven/las/log_aggregator_service.log.$(date +%Y-%m-%d)
```

## Restart procedure

Restart the LAS:

```bash
dh_monit restart log_aggregator_service
```

> [!WARNING]
> Restarting LAS temporarily interrupts log writing from all workers. New workers cannot start until LAS is back online. Running workers may terminate if they cannot buffer log data during the restart.

Verify the restart was successful:

```bash
dh_monit status log_aggregator_service
```

Monitor the log during startup:

```bash
tail -f /var/log/deephaven/las/LogAggregatorService.log.current
```

## Configuring the Log Aggregator Service

The LAS server configuration is managed through the `logAggregatorServers` section of the routing YAML. Client processes enable LAS logging via the `useLogAggregatorService` property. See [Metrics and monitoring](../ops-guide/metrics-and-monitoring.md#log-related-properties) for client configuration and [Data Routing Service YAML format](../configuration/yaml.md#log-aggregator-servers) for server configuration.

## Log Aggregator Service deployment

By default, all nodes in a Deephaven cluster run a LAS. Each LAS writes to its own set of binary log output streams. The [data routing configuration](../configuration/data-routing-overview.md) determines which LAS handles which output streams. For user table data, all sources are directed to the same LAS instance to maintain ordering guarantees.

## Configuration files and locations

**monit configuration:** `/etc/sysconfig/illumon.d/monit/log_aggregator_service.conf`

**Property files:**

- `/etc/sysconfig/illumon.d/resources/iris-common.prop`
- `/etc/sysconfig/illumon.d/resources/log_aggregator_service.prop`

**Binary log output directory:** Defaults to `/var/log/deephaven/binlogs/` (configurable via `binaryLogDir` property; see [Binary log files](../ops-guide/logs/binary-logs.md))

**Process log directory:** `/var/log/deephaven/las/`

## Related documentation

- [Log Aggregator Service overview](../core-components/log-aggregator-service.md)
- [Data Routing Service YAML format](../configuration/yaml.md#log-aggregator-servers)
- [Data routing overview](../configuration/data-routing-overview.md)
- [Metrics and monitoring: log-related properties](../ops-guide/metrics-and-monitoring.md#log-related-properties)
- [Binary log files](../ops-guide/logs/binary-logs.md)
- [Log files reference](../ops-guide/logs/log-files.md)
- [System processes overview](../architecture/architecture-overview.md)
- [Remote Query Dispatcher runbook](runbook-remote-query-dispatcher.md)
- [Configuration Server runbook](runbook-config-server.md)