Export data to CSV or other delimited files

This guide shows how to export table data from Deephaven to CSV files with write_csv, and to other delimited files with pandas.

Tip

You can also export a table to a CSV file from the Deephaven UI with Table Options > Download CSV.

write_csv

Note

The Deephaven server resolves file paths. If you run Deephaven in Docker, the path refers to the container's file system. The default Docker Compose setup mounts the ./data directory of your Deephaven deployment as the /data Docker bind mount in the container. A file written to /data/TrigFunctions.csv appears at ./data/TrigFunctions.csv on your computer. If you run Deephaven without Docker, replace /data with a directory that exists on the server's machine.

Write a table to a CSV file

Let's create a table to export. It contains 100 rows of trigonometric values, made with empty_table and update:

The simplest way to use write_csv is to pass two arguments:

  • The Deephaven source table.
  • The path of the output CSV file.

The first lines of TrigFunctions.csv look like this:

Null values

Null values are common in tables. write_csv writes them as empty fields.

To see this, let's create a table with null values. The example below uses the ternary conditional operator to fill the SinX column with a large number of nulls.

The SinX column contains many null cells. The example below writes this table to a CSV file called TrigFunctionsWithNulls.csv.

The first lines of TrigFunctionsWithNulls.csv show the null SinX values as empty fields:

Column selection

To write only some columns, pass their names in the cols parameter of write_csv. The example below writes the X and CosX columns of the source_with_nulls table created in Null values.

Cosine.csv contains only the X and CosX columns:

Write a ticking table

Deephaven tables can tick, updating in real time as new data arrives. write_csv writes the rows that a table holds when the call runs, so the file is a point-in-time copy of a live table. It reads those rows directly, without taking a snapshot, so a table that updates during the write can produce an inconsistent file. To write a consistent copy, take a snapshot of the table first and write the snapshot.

The example below creates a ticking table with time_table:

A new time table has no rows until Deephaven runs its first update cycle, which happens after the command that creates the table finishes. Writing the table in that same command produces a file with only the header row. Run the write as a separate command, after the table has ticked:

To keep a file up to date as a table ticks, see How can I periodically write a ticking table to disk?.

Write other delimited files with pandas

write_csv always separates fields with commas. It has no parameter for a different delimiter.

To write a file with a different delimiter, convert the table to a pandas DataFrame with to_pandas and call the DataFrame's to_csv method with the sep argument. The example below writes the source table created in Write a table to a CSV file as a tab-separated file:

to_pandas copies the whole table into memory, so for a large table, select only the rows and columns you need first.