Package io.deephaven.iceberg.util

Class IcebergTools

java.lang.Object
io.deephaven.iceberg.util.IcebergTools
public final class IcebergTools extends Object
Tools for accessing tables in the Iceberg table format.

  • Constructor Details

    • IcebergTools

      public IcebergTools()

  • Method Details

    • createAdapter

      public static IcebergCatalogAdapter createAdapter(org.apache.iceberg.catalog.Catalog catalog)

    • createAdapter

      public static IcebergCatalogAdapter createAdapter(@Nullable @Nullable String name, @NotNull @NotNull Map<String,String> properties)

      Create an Iceberg catalog adapter for an Iceberg catalog created from configuration properties. These properties map to the Iceberg catalog Java API properties and are used to create the catalog and file IO implementations.

      The minimal set of properties required to create an Iceberg catalog are:

      • "catalog-impl" or "type" - the Java catalog implementation to use. When providing "catalog-impl", the implementing Java class should be provided (e.g. "org.apache.iceberg.rest.RESTCatalog" or "org.apache.iceberg.aws.glue.GlueCatalog"). Choices for "type" include "hive", "hadoop", "rest", "glue", "nessie", "jdbc".

      Other common properties include:

      • "uri" - the URI of the catalog.
      • "warehouse" - the location of the data warehouse.
      • "client.region" - the region of the AWS client.
      • "s3.access-key-id" - the S3 access key for reading files.
      • "s3.secret-access-key" - the S3 secret access key for reading files.
      • "s3.endpoint" - the S3 endpoint to connect to.

      Additional properties for the specific catalog should also be included, such as as S3-specific properties for authentication or endpoint overriding.

      Parameters:
      name - the name of the catalog; if omitted, the catalog URI will be used to generate a name
      properties - a map containing the Iceberg catalog properties to use
      Returns:
      the Iceberg catalog adapter

    • createAdapter

      public static IcebergCatalogAdapter createAdapter(@Nullable @Nullable String name, @NotNull @NotNull Map<String,String> properties, @NotNull @NotNull Map<String,String> hadoopConfig)

      Create an Iceberg catalog adapter for an Iceberg catalog created from configuration properties. These properties map to the Iceberg catalog Java API properties and are used to create the catalog and file IO implementations.

      The minimal set of properties required to create an Iceberg catalog are:

      • "catalog-impl" or "type" - the Java catalog implementation to use. When providing "catalog-impl", the implementing Java class should be provided (e.g. "org.apache.iceberg.rest.RESTCatalog" or "org.apache.iceberg.aws.glue.GlueCatalog"). Choices for "type" include "hive", "hadoop", "rest", "glue", "nessie", "jdbc".

      Other common properties include:

      • "uri" - the URI of the catalog.
      • "warehouse" - the location of the data warehouse.
      • "client.region" - the region of the AWS client.
      • "s3.access-key-id" - the S3 access key for reading files.
      • "s3.secret-access-key" - the S3 secret access key for reading files.
      • "s3.endpoint" - the S3 endpoint to connect to.

      Additional properties for the specific catalog should also be included, such as as S3-specific properties for authentication or endpoint overriding.

      Parameters:
      name - the name of the catalog; if omitted, the catalog URI will be used to generate a name
      properties - a map containing the Iceberg catalog properties to use
      hadoopConfig - a map containing Hadoop configuration properties to use
      Returns:
      the Iceberg catalog adapter