Create static tables
Deephaven often reads table data from Parquet, Kafka, or other external sources, but it can also generate tables from scratch. Static tables hold fixed data that does not change after creation. This guide covers two simple functions for creating them: empty_table and new_table. It shows how to use these functions to create static tables and columns, and how to add data to those tables. To create a table that updates in real time, see Create a time table, Write data to an in-memory, real-time table, Create and use input tables, and Replay data from static tables.
empty_table
The empty_table function takes a single argument — an int representing the number of rows in the new table. The resulting table has no columns and the specified number of rows. In the following example, we create a table with 10 rows and no columns:
Calling empty_table on its own generates a table with no data. You can add columns and data with update or another selection method, either in the same line that creates the table or at any time afterward.
In the following example, we create a table with 10 rows and a single column X with values 0 through 9 by using the special variable i to represent the row index. Then, we update the table again to add a column Y with values equal to X squared:
The Create new columns in a table section shows more ways to add columns.
new_table
Deephaven's new_table function creates a new table that you populate with data. It accepts either a list of column objects or a dictionary that maps column names to data. This section covers column objects first, including the column types they support and array columns, then creating a table from a dictionary.
A column object is an InputColumn, which stores a column's name, type, and data. Functions such as string_col and int_col create them. The Column types section lists these functions.
The following query creates a new table with a String column and an int column from a list of column objects:
Column types
The following functions create columns of common types:
| Data type | Function |
|---|---|
Boolean | bool_col |
byte | byte_col |
char | char_col |
java.time.Instant | datetime_col |
double | double_col |
float | float_col |
int | int_col |
java.lang.Object | jobj_col |
long | long_col |
| Python Object | pyobj_col |
short | short_col |
String | string_col |
To create a column of a type not in this list, such as an array column, see Array columns.
Array columns
new_table can also create array columns, where each cell holds an array. Typed functions such as int_col and string_col can't create these. Instead, construct an InputColumn yourself with an array type from the deephaven.dtypes module, such as dtypes.int32_array.
The following example creates a new table with a single integer array column. dtypes.array converts a NumPy array to a Java int array. The column's data is a list that holds that one array, so the table has one row.
Create a table from a dictionary
When you pass a dictionary, new_table first converts it to a pandas DataFrame, so pandas infers each column's type from its data. In the following example, the Name column is a String column, Math is a long column, and Gpa is a double column:
Create new columns in a table
Selection methods and formulas work together to create new columns. The selection methods are select, view, update, update_view, and lazy_update. Selection methods and formulas each have their own job:
- The selection method determines which columns appear in the output table. It also determines whether their values are computed immediately and stored in memory, or computed on demand when they are read.
- The formulas are the recipes for computing the cell values.
In the following example, we use a table of student test results. Using update, we create a new Total column containing the sum of each student's math, science, and art scores, and an Average column computed from Total.
A formula can use a column created by an earlier formula in the same call, as Average uses Total here. The output table also keeps every column from the source table.
select, by contrast, includes only the columns you list. The following example keeps the Name column and adds a Total column:
The formulas you pass to update and the other selection methods can use the full Deephaven Query Language (DQL), including mathematical operations, built-in functions, comparison and conditional operators, Java functions, and user-defined functions. In the following example, we create a table with 100 rows, then create four columns:
The following example creates a table with two integer columns. Then, it updates the table to add a new column X via a formula that uses a variable, a Python function, an auto-imported Java function, and various operators:
For more about DQL, see the Query string overview and Select and create columns.