adapter

The adapter method creates an Iceberg catalog adapter from configuration properties. It is a general-purpose constructor method that can be used to create an adapter for a wide variety of Iceberg catalog types.

Syntax

adapter(
    name: str = None,
    properties: Dict[str, str] = None,
    hadoop_config: Dict[str, str] = None
)

Parameters

ParameterTypeDescription
namestr

A descriptive name of the catalog. If omitted, the catalog name is inferred from the catalog URI.

propertiesDict[str, str]

The properties of the catalog. Two properties must be given when creating a catalog adapter:

  • catalog-impl or type
    • catalog-impl is the Java catalog implementation class name, such as:
      • org.apache.iceberg.aws.glue.GlueCatalog
      • org.apache.iceberg.rest.RESTCatalog
    • type
      • type is the type of adapter to create. The following types are supported:
        • hive
        • hadoop
        • rest
        • glue
        • nessie
        • jdbc
  • uri
    • The URI of the catalog

Additional optional properties include:

  • warehouse - the root path of the data warehouse.
  • client.region - The region of the AWS client.
  • s3.access-key-id - The access key ID for the S3 client.
  • s3.secret-access-key - The secret access key for the S3 client.
  • s3.endpoint - The endpoint for the S3 client.
  • io-impl - The implementation class for the I/O operations, such as org.apache.iceberg.aws.s3.S3FileIO for S3-compatible storage.
hadoop_configDict[str, str]

Hadoop configuration properties for the catalog.

Returns

An IcebergCatalogAdapter.

Examples

The following example creates a catalog adapter using a local MinIO instance and REST catalog:

from deephaven.experimental import iceberg

local_adapter = iceberg.adapter(
    name="generic-adapter",
    properties={
        "type": "rest",
        "uri": "http://rest:8181",
        "client.region": "us-east-1",
        "s3.access-key-id": "admin",
        "s3.secret-access-key": "password",
        "s3.endpoint": "http://minio:9000",
        "io-impl": "org.apache.iceberg.aws.s3.S3FileIO",
    },
)

The following example creates an Iceberg catalog adapter that connects to an AWS Glue catalog:

from deephaven.experimental import s3, iceberg

cloud_adapter = iceberg.adapter(
    name="generic-adapter",
    properties={
        "type": "glue",
        "uri": "s3://lab-warehouse/sales",
    },
)