Database Event Logs

Some Deephaven components can be configured to log their events to the Deephaven database tables in place of text files. This drastically improves the ability to search through logs and find related events, especially for query workers where a user would otherwise need access to the server running the worker to view its log. The full power of the Deephaven Query Language is available to view these table-stored event logs, instead of only the basic text processing utilities like grep, awk, sed, etc.

Process Event logs

The Process Event Log (PEL) contains information about query worker processes. If you want to see the logs for worker_3a641327, you can use the following query to view them:

pel = db.liveTable("DbInternal", "ProcessEventLog").where("Date=today()", "Process=`worker_3a641327`")

pel = db.live_table("DbInternal", "ProcessEventLog").where(
    ["Date=today()", "Process=`worker_3a641327`"]
)

Tip

You can also find the worker ID in the header of a worker console.

Note

Non-privileged users can only see records for which the EffectiveUser matches their username in the ProcessEventLog table.

The ProcessEventLog records logs for configured Deephaven processes, including columns for Log Entry, Log Level, Date, Time, and Effective User.

For more on using the ProcessEventLog to troubleshoot queries, see Monitor and Troubleshoot queries with the ProcessEventLog.

It's best practice to filter a table on its partitioning column(s) before doing any other operations. By default, internal tables are partitioned on the Date column. Partitioning columns are set in schema files.

The following query shows all events for today, as well as all ERROR level events for today:

// Show all events for today
today = today()
pel = db.liveTable("DbInternal", "ProcessEventLog").where("Date=today")

// Show all ERROR Level events for today
pelErrors = db.liveTable("DbInternal", "ProcessEventLog").where("Date=today", "Level=`ERROR`")

# Show all events for today
pel = db.live_table("DbInternal", "ProcessEventLog").where("Date=today()")

# Show all ERROR Level events for today
pel_errors = db.live_table("DbInternal", "ProcessEventLog").where(
    ["Date=today()", "Level=`ERROR`"]
)

Audit Event logs

The AuditEventLog captures all audit events such as user login attempts, process startup events, and more. Like before, it's best practice to filter the AuditEventLog on date. The following query shows all events for today:

// Show all Audit events for today
ael = db.liveTable("DbInternal",  "AuditEventLog").where("Date=today()")

# Show all Audit events for today
ael = db.live_table("DbInternal", "AuditEventLog").where("Date=today()")

Note

Non-priveleged users can only see records for which the EffectiveUser matches their username in the AuditEventLog table.