--- title: ProcessMetricsLogCoreV2 --- The `ProcessMetricsLogCoreV2` table captures internal metrics that were previously written to the `stats.log` CSV. The table is disabled by default. To enable the `ProcessMetricsLogCoreV2` table for all processes, set the following property to `true`: ```properties IrisLogDefaults.writeDatabaseProcessMetrics=true ``` > [!CAUTION] > Enabling this for all processes will generate a large amount of data, which can impact performance. It is recommended that this be enabled only for debugging purposes or for specific processes where detailed metrics are required. To enable this for a single Persistent Query or Code Studio: 1. Click the **Show Advanced** dropdown (for a Persistent Query, it is in the **Settings** tab). 2. Add `-DIrisLogDefaults.writeDatabaseProcessMetrics=true` to the `Extra JVM Arguments` field. ## Columns | Column Name | Column Type | Description | | ------------------- | ----------- | --------------------------------------------------------------------------------------------------------------------------------- | | `Date` | String | The date on which the information was generated. This is the partitioning column. | | `AuthenticatedUser` | String | The authenticated user (e.g., the user who logged in via password, SAML, etc.) that owns the worker. | | `EffectiveUser` | String | The effective user (e.g., the "operate as" user if the authenticated user is using the "operate as" option) that owns the worker. | | `ProcessInfoId` | String | The `ProcessInfoID` of the process that generated this event. | | `Timestamp` | DateTime | The timestamp for this event. | | `Name` | String | The name of the metric. | | `Interval` | String | The timing interval for the metric. | | `Type` | String | The type of the metric. | | `N` | long | Data specific to the metric type. | | `Sum` | long | Data specific to the metric type. | | `Last` | long | Data specific to the metric type. | | `Min` | long | Data specific to the metric type. | | `Max` | long | Data specific to the metric type. | | `Avg` | long | Data specific to the metric type. | | `Sum2` | long | Data specific to the metric type. | | `Stdev` | long | Data specific to the metric type. | ## Understanding the statistical fields The `ProcessMetricsLogCoreV2` table records statistical data about various system metrics over time. Each row represents a snapshot of a metric during a specific time interval. The statistical fields provide different views of the metric's behavior during that interval: - **`N`** - The number of observations or events recorded during the interval. - **`Sum`** - The total sum of all values recorded during the interval. - **`Last`** - The most recent value recorded in the interval. - **`Min`** - The minimum (smallest) value observed during the interval. - **`Max`** - The maximum (largest) value observed during the interval. - **`Avg`** - The average value across all observations (calculated as Sum / N). - **`Sum2`** - The sum of squares of all values (used for variance calculations). - **`Stdev`** - The standard deviation of values during the interval. ### Example: Counter metrics For a counter metric like `Memory-GC-G1.YoungGenCollectionCount`: - `N=5` means 5 garbage collection events occurred during the interval. - `Sum=250` means a total of 250 milliseconds spent in GC across those events. - `Last=60` means the most recent GC took 60 milliseconds. - `Min=30` means the shortest GC was 30 milliseconds. - `Max=80` means the longest GC was 80 milliseconds. - `Avg=50` means the average GC duration was 50 milliseconds. To view counter metrics: ```python # Get all Memory-GC-G1 metrics for today gc_metrics = ( db.live_table("DbInternal", "ProcessMetricsLogCoreV2") .where(["Date=today()", "Name.startsWith(`Memory-GC.G1`)"]) .view(["Timestamp", "Name", "Type", "N", "Sum", "Last", "Min", "Max", "Avg"]) ) # Get all counter-type metrics counter_metrics = ( db.live_table("DbInternal", "ProcessMetricsLogCoreV2") .where(["Date=today()", "Type=`counter`"]) .view(["Timestamp", "Name", "N", "Sum", "Last", "Min", "Max", "Avg"]) ) ``` ```groovy // Get all Memory-GC-G1 metrics for today gc_metrics = db.liveTable("DbInternal", "ProcessMetricsLogCoreV2").where( "Date=today()", "Name.startsWith(`Memory-GC.G1`)" ).view("Timestamp", "Name", "Type", "N", "Sum", "Last", "Min", "Max", "Avg") // Get all counter-type metrics counter_metrics = db.liveTable("DbInternal", "ProcessMetricsLogCoreV2").where( "Date=today()", "Type=`counter`" ).view("Timestamp", "Name", "N", "Sum", "Last", "Min", "Max", "Avg") ``` ### Example: State metrics For a state metric like `Memory-Heap.Used`: - `N=10` means 10 samples were taken during the interval. - `Last=4294967296` means the current heap usage is 4,294,967,296 bytes (4 GB). - `Min=3221225472` means the minimum heap usage during the interval was ~3 GB. - `Max=5368709120` means the maximum heap usage during the interval was ~5 GB. - `Avg` shows the average heap usage across all samples. To view state metrics: ```python # Get all memory metrics for today memory_metrics = ( db.live_table("DbInternal", "ProcessMetricsLogCoreV2") .where(["Date=today()", "Name.startsWith(`Memory-`)"]) .view(["Timestamp", "Name", "Last", "Min", "Max", "Avg"]) ) # Focus on heap usage specifically with conversion to GB heap_usage = ( db.live_table("DbInternal", "ProcessMetricsLogCoreV2") .where(["Date=today()", "Name=`Memory-Heap.Used`"]) .update("LastGB=Last/1073741824.0") .view(["Timestamp", "LastGB", "Min", "Max"]) ) ``` ```groovy // Get all memory metrics for today memory_metrics = db.liveTable("DbInternal", "ProcessMetricsLogCoreV2").where( "Date=today()", "Name.startsWith(`Memory-`)" ).view("Timestamp", "Name", "Last", "Min", "Max", "Avg") // Focus on heap usage specifically with conversion to GB heap_usage = db.liveTable("DbInternal", "ProcessMetricsLogCoreV2").where( "Date=today()", "Name=`Memory-Heap.Used`" ).update("LastGB=Last/1073741824.0").view("Timestamp", "LastGB", "Min", "Max") ``` ## Metric types The `Type` column indicates the category of metric being recorded. Deephaven uses two primary metric types: ### Counter A **counter** tracks cumulative events or operations over time. Counter metrics are used to measure things that accumulate, such as: - Number of read or write operations - Number of bytes transferred - Number of operations completed For counter metrics, the statistical fields track the individual events: - **`N`** - The count of events that occurred during the interval. - **`Sum`** - The total accumulated value across all events (e.g., total bytes). - **`Last`** - The most recent individual event value. - **`Min`/`Max`** - The smallest/largest individual event values. - **`Avg`** - The average event value (Sum / N). Example: `Memory-GC-G1.YoungGenCollectionCount` with `Type='counter'` tracks individual garbage collection events. ### State A **state** metric represents a sampled measurement of current system state. State metrics are used to measure things that fluctuate, such as: - Current memory usage - Current thread counts - Current queue depths For state metrics, the statistical fields track the sampled values: - **`N`** - The number of samples taken during the interval - **`Sum`** - The sum of all sampled values - **`Last`** - The most recent sampled value (typically the most useful for state metrics) - **`Min`/`Max`** - The minimum/maximum values observed during sampling - **`Avg`** - The average value across all samples (Sum / N) Example: `Memory-Heap.Used` with `Type='state'` tracks the current heap memory usage at each sample point. ## Common metrics Deephaven processes log various metrics, including: - **Memory-Heap.Used** - JVM heap memory usage (state) - **Memory-NonHeap.Used** - JVM non-heap memory usage, e.g., metaspace (state) - **Memory-Direct.Used** - Direct buffer memory usage (state) - **Memory-GC-G1.\*** - G1 garbage collection metrics (counter) - **Thread.\*** - Thread-related metrics (state) ## Querying metrics To analyze `ProcessMetricsLogCoreV2` data, you can query the table and filter by specific metric names: ```python # Get all available metric names for today metric_names = ( db.live_table("DbInternal", "ProcessMetricsLogCoreV2") .where("Date=today()") .select_distinct("Name") .sort("Name") ) # Get memory usage over time memory_metrics = ( db.live_table("DbInternal", "ProcessMetricsLogCoreV2") .where(["Date=today()", "Name.startsWith(`Memory-`)"]) .view(["Timestamp", "Name", "Last", "Min", "Max", "Avg"]) ) ``` ```groovy // Get all available metric names for today metric_names = db.liveTable("DbInternal", "ProcessMetricsLogCoreV2").where( "Date=today()" ).selectDistinct("Name").sort("Name") // Get memory usage over time memory_metrics = db.liveTable("DbInternal", "ProcessMetricsLogCoreV2").where( "Date=today()", "Name.startsWith(`Memory-`)" ).view("Timestamp", "Name", "Last", "Min", "Max", "Avg") ``` ## Related documentation - [Internal tables overview](./internal-tables.md) - [`AuditEventLog`](./audit-event-log.md) - [`PersistentQueryConfigurationLog`](./persistent-query-configuration-log.md) - [`PersistentQueryStateLog`](./persistent-query-state-log.md) - [`ProcessEventLogIndex`](./process-event-log-index.md) - [`ProcessEventLog`](./process-event-log.md) - [`ProcessInfo`](./process-info.md) - [`QueryOperationPerformanceLogCoreV2Index`](./query-operation-performance-log-index.md) - [`QueryOperationPerformanceLogCoreV2`](./query-operation-performance-log.md) - [`QueryPerformanceLogCoreV2`](./query-performance-log.md) - [`QueryUserAssignmentLog`](./query-user-assignment-log.md) - [`ResourceUtilization`](./resource-utilization.md) - [`ServerStateLogIndex`](./server-state-log-index.md) - [`ServerStateLog`](./server-state-log.md) - [`UpdatePerformanceLogCoreV2Index`](./update-performance-log-core-index.md) - [`UpdatePerformanceLogCoreV2`](./update-performance-log-core.md) - [`WorkspaceDataSnapshot`](./workspace-data-snapshot.md) - [`WorkspaceData`](./workspace-data.md)