IcebergCatalogAdapter
The IcebergCatalogAdapter
class provides an interface for interacting with Iceberg catalogs. It allows listing namespaces, tables, and snapshots, as well as reading Iceberg tables into Deephaven tables.
Constructors
An IcebergCatalogAdapter
is constructed using one of the following methods:
Methods
namespaces
- Returns a table of Iceberg namespaces that belong to a given namespace. If no namespace is given, a table of the top-level namespaces is returned.read_table
- Reads a table from an Iceberg catalog using provided instructions.snapshots
- Returns a table of snapshots for a given Iceberg table.tables
- Returns a table of Iceberg tables in a given namespace.
Examples
The following example creates an IcebergCatalogAdapter
using an AWS glue catalog. The catalog is then used to get the top-level namespaces in the catalog and the tables in the sales
namespace:
from deephaven.experimental import iceberg
cloud_adapter = iceberg.adapter_aws_glue(
name="aws-iceberg",
catalog_uri="s3://lab-warehouse/sales",
warehouse_location="s3://lab-warehouse/sales",
)
namespaces = cloud_adapter.namespaces()
sales_tables = cloud_adapter.tables("sales")
The following example creates an IcebergCatalogAdapter
using a local MinIO instance and a REST catalog. The catalog is then used to get the snapshots for the sales_multi
table in the sales
namespace. The sales.sales_multi
table is then read into a Deephaven table:
from deephaven.experimental import iceberg
local_adapter = iceberg.adapter_s3_rest(
name="minio-iceberg",
catalog_uri="http://rest:8181",
warehouse_location="s3a://warehouse/wh",
region_name="us-east-1",
access_key_id="admin",
secret_access_key="password",
end_point_override="http://minio:9000",
)
sales_multi_snapshots = local_adapter.snapshots("sales.sales_multi")
sales_multi = local_adapter.read_table("sales.sales_multi")