Skip to main content
Version: Python

How can I periodically write a ticking table to disk?

It's common to have queries on ticking tables where it's prudent to write that data to disk periodically. There are a few ways to do this.

Python timers

There are several ways to periodically call functions in Python. The following example uses Python's threading.Timer to write a ticking table to disk every 30 seconds.

from deephaven import time_table
from deephaven import write_csv
import threading
import traceback
import time

# The ticking table to write to disk
my_data = time_table("PT0.2s").update(["X = i", "Y = randomDouble(0, 100)"])


# A function that runs another function periodically
def run_periodically(delay, task):
next_time = time.time() + delay
while True:
time.sleep(max(0, next_time - time.time()))
try:
task()
except Exception:
traceback.print_exc()
next_time += (time.time() - next_time) // delay * delay + delay


# The function to be called by `run_periodically`
def write_to_disk() -> None:
write_csv(my_data, "/data/my_data.csv")


# A threading Timer to write to disk every 30 seconds
t = threading.Thread(target=lambda: run_periodically(30, write_to_disk)).start()

Time table

You can also just use a time table to trigger writes. For instance, if you want to write every 30 seconds, have a time table tick every 30 seconds and call a Python function that writes the ticking table of interest to disk.

from deephaven import time_table, ring_table
from deephaven import write_csv

# The ticking table to write to disk
my_data = time_table("PT0.2s").update(
["X = randomDouble(0, 100)", "Y = randomInt(-100, 100)", "Z = randomBool()"]
)


# A function to write data to disk - returns True if successful, False otherwise
def write_to_disk() -> bool:
try:
write_csv(my_data_ring, f"/data/my_data.csv")
except:
return False
return True


# Your write trigger table
write_trigger = time_table("PT30s").update("WriteSuccessful = write_to_disk()")