Track processing time and measure latency

This guide explains how to track when Deephaven processes row modifications, which is useful for measuring end-to-end latency in real-time data pipelines.

Note

This feature requires using jpy to access the Java API directly. See the Groovy documentation for a full explanation of the concepts.

Problem

By default, Deephaven optimizes formula evaluation by only recomputing values when their input columns change. A formula like ProcessTime = now() evaluates once when the row is created but does not re-evaluate when other columns change — because now() has no column dependencies.

This matters when you receive data from external sources (like Kafka) that include a source timestamp. You want to compare that timestamp against when Deephaven processed the update — but a simple now() formula won't re-evaluate when the row is modified.

Solution

Use SelectColumnFactory.getExpression with withRecomputeOnModifiedRow to force the formula to re-evaluate every time a row is modified.

Since there's no native Python wrapper for this feature, use jpy.get_type to access the Java class. The jpy-obtained SelectColumn isn't recognized by the Python wrapper's type checking, so you need to call j_table.update directly and wrap the result.