IcebergTableAdapter

The IcebergTableAdapter class provides an interface for interacting with Iceberg tables. It enables listing snapshots, retrieving table definitions, and reading Iceberg tables into Deephaven tables.

Constructors

An IcebergTableAdapter is constructed with IcebergCatalogAdapter.load_table.

Methods

  • snapshots: Returns information on the snapshots of a particular Iceberg table.
  • definition: Returns the Iceberg table definition as a Deephaven table. Can optionally specify custom IcebergReadInstructions to perform column renames, give custom definitions, and specify data instructions.
  • table: Reads the Iceberg table using the provided instructions. A snapshot ID can be provided to read a specific snapshot.
  • table_writer: Create a new IcebergTableWriter using the provided writer options. This method performs schema validation to ensure that the provided table definition is compatible with the Iceberg table schema.

Examples

The following example creates an IcebergTableAdapter called iceberg_taxis from an IcebergCatalogAdapter. The catalog adapter is created using a local MinIO instance and a REST catalog. The table adapter is then used to get a table of all available snapshots, load the table definition, and load the taxis Iceberg table into a Deephaven table:

from deephaven.experimental import iceberg

local_adapter = iceberg.adapter_s3_rest(
    name="minio-iceberg",
    catalog_uri="http://rest:8181",
    warehouse_location="s3a://warehouse/wh",
    region_name="us-east-1",
    access_key_id="admin",
    secret_access_key="password",
    end_point_override="http://minio:9000",
)

iceberg_taxis = local_adapter.load_table("taxis")
taxi_snapshots = iceberg_taxis.snapshots()
taxis_definition = iceberg_taxis.definition()
taxis = iceberg_taxis.table()