deephaven.pandas

This module supports the conversion between Deephaven tables and pandas DataFrames.

to_pandas(table, cols=None, dtype_backend='numpy_nullable', conv_null=True)[source]

Produces a pandas DataFrame from a table.

Note that the entire table is going to be cloned into memory, so the total number of entries in the table should be considered before blindly doing this. For large tables, consider using the Deephaven query language to select a subset of the table before using this method.

Parameters:

  • table (Table) – the source table

  • cols (List[str]) – the source column names, default is None which means include all columns

  • dtype_backend (str) – which dtype_backend to use, e.g. whether a DataFrame should have NumPy arrays, nullable dtypes are used for all dtypes that have a nullable implementation when “numpy_nullable” is set, pyarrow is used for all dtypes if “pyarrow” is set. None means Numpy backed DataFrames with no nullable dtypes. Both “numpy_nullable” and “pyarrow” automatically convert Deephaven nulls to Pandas NA and enable Pandas extension types. Extension types are needed to support types beyond NumPy’s type system. Extension types support operations such as properly mapping Java Strings to Python strings. default is “numpy_nullable”.

  • conv_null (bool) – when dtype_backend is set to None, whether to check for Deephaven nulls in the data and automatically replace them with pd.NA. default is True.

Return type:

DataFrame

Returns:

a pandas DataFrame

Raises:

DHError –

to_table(df, cols=None, infer_objects=True)[source]

Creates a new table from a pandas DataFrame.

Parameters:

  • df (DataFrame) – the pandas DataFrame instance

  • cols (List[str]) – the dataframe column names, default is None which means including all columns in the DataFrame

  • infer_objects (bool) – whether to infer the best possible types for columns of the generic ‘object’ type in the DataFrame before creating the table, default is True. When True, pandas convert_dtypes() method is called to perform the conversion. Note that any conversion will make a copy of the data.

Return type:

Table

Returns:

a Deephaven table

Raises:

DHError –