Skip to main content
Version: Python

IcebergCatalogAdapter

The IcebergCatalogAdapter class provides an interface for interacting with Iceberg catalogs. It allows listing namespaces, tables, and snapshots, as well as reading Iceberg tables into Deephaven tables.

Constructors​

An IcebergCatalogAdapter is constructed using one of the following methods:

Methods​

  • namespaces - Returns a table of Iceberg namespaces that belong to a given namespace. If no namespace is given, a table of the top-level namespaces is returned.
  • read_table - Reads a table from an Iceberg catalog using provided instructions.
  • snapshots - Returns a table of snapshots for a given Iceberg table.
  • tables - Returns a table of Iceberg tables in a given namespace.

Examples​

The following example creates an IcebergCatalogAdapter using an AWS glue catalog. The catalog is then used to get the top-level namespaces in the catalog and the tables in the sales namespace:

from deephaven.experimental import iceberg

cloud_adapter = iceberg.adapter_aws_glue(
    name="aws-iceberg",
    catalog_uri="s3://lab-warehouse/sales",
    warehouse_location="s3://lab-warehouse/sales",
)

namespaces = cloud_adapter.namespaces()
sales_tables = cloud_adapter.tables("sales")

The following example creates an IcebergCatalogAdapter using a local MinIO instance and a REST catalog. The catalog is then used to get the snapshots for the sales_multi table in the sales namespace. The sales.sales_multi table is then read into a Deephaven table:

from deephaven.experimental import iceberg

local_adapter = iceberg.adapter_s3_rest(
    name="minio-iceberg",
    catalog_uri="http://rest:8181",
    warehouse_location="s3a://warehouse/wh",
    region_name="us-east-1",
    access_key_id="admin",
    secret_access_key="password",
    end_point_override="http://minio:9000",
)

sales_multi_snapshots = local_adapter.snapshots("sales.sales_multi")
sales_multi = local_adapter.read_table("sales.sales_multi")