Replay data from static tables

This guide shows you how to replay historical data as if it were live. A replayer adds each row of a static table to a ticking table when its replay clock reaches the row's timestamp. A ticking table is a table whose rows change as new data arrives. The replay clock starts at a start time you choose and advances in real time. Replaying pre-recorded data is useful for learning and testing, and for fields that integrate historical data into real-time analysis, such as machine learning, validation, modeling, simulation, and forecasting. This guide covers replaying a table, replaying a table that has no timestamp column, and replaying multiple tables in sync.

Prepare a historical data table

To replay historical data, you need a table with a timestamp column of type Instant. The timestamp column must not contain null values, and its values must not decrease from one row to the next. Sort the table on that column with sort first if needed. If your table has no timestamp column, see Replay a table with no timestamp column.

Let's grab a table from Deephaven's examples repository. This example uses data from a 100 km bike ride in a file called metriccentury.csv. Its Time column holds the timestamps, already in ascending order.

Replay a table

Load the metricCentury table from Prepare a historical data table first. Then replay it with the following steps:

  • Import Replayer.
  • Choose a start time and an end time for the replay.
    • The replay clock starts at the start time and advances in real time. Replay stops at the end time.
    • To replay the whole table, use its first and last timestamps, as this example does with the first and last values in Time.
  • Create the replayer with those start and end times.
    • The Replayer constructor takes Instant values, so use parseInstant to convert date-time strings.
  • Call replay with the source table and the name of its timestamp column.
    • It returns a new ticking table. The table starts with every source row whose timestamp is at or before the start time. In this example, that's the first row.
    • The replayer adds each remaining row to the new table when the replay clock reaches that row's timestamp.
    • The source table itself doesn't change.
    • The timestamp column must meet the requirements in Prepare a historical data table.
  • Call start to start replaying data.

The ride lasts about five and a half hours, so this replay takes that long to finish. Call shutdown to stop replaying early.

Replay a table with no timestamp column

Some historical data tables don't have a timestamp column.

In such a case, add a timestamp column of type Instant.

Replay irisWithDatetimes with a Replayer, the same way as in Replay a table. Pass the new Timestamp column as the timestamp column. This example reuses startTime from the previous block as the replay start time. The table has 150 rows one second apart, so an end time 2 minutes 30 seconds later replays every row.

Replay multiple tables

Real-time applications in Deephaven commonly involve more than one ticking table. A single replayer can replay multiple tables at the same time. All of them share the same replay clock, which starts at the replayer's start time and advances in real time. Each table's rows appear when that clock reaches their timestamps, so the replayed tables stay in sync.

The following code creates two tables with timestamps that overlap.

To replay multiple tables with the same replayer, call replay once per table before start.