IcebergReadInstructions

The IcebergReadInstructions class specifies the instructions for reading Iceberg tables into Deephaven. These include special instructions for loading files from cloud storage, the update mode, and the snapshot to read.

Note

The table_definition and column_renames parameters are deprecated and have no effect. Passing either one only raises a DeprecationWarning. To choose columns, set their types, or map Deephaven column names to Iceberg fields, pass an UnboundResolver to IcebergCatalogAdapter.load_table instead.

Syntax

Parameters

ParameterTypeDescription
table_definitionUnion[Dict[str, DType], List[Column]]

Deprecated. This parameter has no effect.

data_instructionsS3Instructions

Special instructions for reading data files from a non-local file system, such as S3. If not given, the data instructions are derived from the catalog.

column_renamesDict[str, str]

Deprecated. This parameter has no effect.

update_modeIcebergUpdateMode

The update mode for the table. Options include:

  • IcebergUpdateMode.static(): Specifies that the Iceberg table should be loaded once and not refreshed.
  • IcebergUpdateMode.manual_refresh(): Specifies that the Iceberg table should be loaded once and refreshed manually.
  • IcebergUpdateMode.auto_refresh(): Specifies that the Iceberg table should be loaded once and refreshed automatically. The default refresh interval is 60 seconds, but can be changed with the auto_refresh_ms input parameter.
snapshot_idint

The snapshot ID to read. If not given, the most recent snapshot ID is used.

ignore_resolving_errorsbool

Whether to ignore unexpected errors when resolving columns. If True, Deephaven reads null values for a column in any data file where the column should be present but can't be resolved. These errors can be a sign of an incorrect resolver or name mapping, or of an Iceberg metadata or data problem. The default is False.

Methods

None.

Constructors

An IcebergReadInstructions is constructed directly from the class.

Examples

The following example creates four IcebergReadInstructions objects. The first is for static Iceberg tables, the second is for Iceberg tables that can be manually refreshed, and the third and fourth are for Iceberg tables that will be refreshed automatically. The third uses the default value of 60 seconds, whereas the fourth sets the interval to 30 seconds.

The following example creates an IcebergReadInstructions object that tells a catalog adapter about the region, access information, and endpoint for reading Iceberg tables from S3 cloud storage: