Package io.deephaven.iceberg.base
Class IcebergUtils
java.lang.Object
io.deephaven.iceberg.base.IcebergUtils
-
Nested Class Summary
Modifier and TypeClassDescriptionstatic final class
-
Constructor Summary
-
Method Summary
Modifier and TypeMethodDescriptionstatic Stream<org.apache.iceberg.DataFile>
allDataFiles
(@NotNull org.apache.iceberg.Table table, @NotNull org.apache.iceberg.Snapshot snapshot) Get a stream of allDataFile
objects from the givenTable
andSnapshot
.static Stream<org.apache.iceberg.ManifestFile>
allManifestFiles
(@NotNull org.apache.iceberg.Table table, @NotNull org.apache.iceberg.Snapshot snapshot) Get a stream of allManifestFile
objects from the givenTable
andSnapshot
.static Type<?>
convertToDHType
(@NotNull org.apache.iceberg.types.Type icebergType) Convert an Iceberg data type to a Deephaven type.static org.apache.iceberg.types.Type
convertToIcebergType
(Class<?> columnType) Convert a Deephaven type to an Iceberg type.static boolean
createNamespaceIfNotExists
(@NotNull org.apache.iceberg.catalog.Catalog catalog, @NotNull org.apache.iceberg.catalog.Namespace namespace) static org.apache.iceberg.PartitionSpec
createPartitionSpec
(@NotNull org.apache.iceberg.Schema schema, @NotNull Iterable<String> partitionColumnNames) static IcebergUtils.SpecAndSchema
createSpecAndSchema
(@NotNull TableDefinition tableDefinition) static boolean
dropNamespaceIfExists
(@NotNull org.apache.iceberg.catalog.Catalog catalog, @NotNull org.apache.iceberg.catalog.Namespace namespace) static <T> Stream<T>
toStream
(org.apache.iceberg.io.CloseableIterable<T> iterable) Convert aCloseableIterable
to aStream
that will close the iterable when the stream is closed.static void
verifyPartitioningColumns
(org.apache.iceberg.PartitionSpec tablePartitionSpec, TableDefinition tableDefinition) Check that all the partitioning columns from the partition spec are present in the Table Definition.static void
verifyRequiredFields
(org.apache.iceberg.Schema tableSchema, TableDefinition tableDefinition) Check that all required fields are present in the table definition
-
Constructor Details
-
IcebergUtils
public IcebergUtils()
-
-
Method Details
-
allDataFiles
public static Stream<org.apache.iceberg.DataFile> allDataFiles(@NotNull @NotNull org.apache.iceberg.Table table, @NotNull @NotNull org.apache.iceberg.Snapshot snapshot) Get a stream of allDataFile
objects from the givenTable
andSnapshot
.- Parameters:
table
- TheTable
to retrieve data files for.snapshot
- TheSnapshot
to retrieve data files from.- Returns:
- A stream of
DataFile
objects.
-
allManifestFiles
public static Stream<org.apache.iceberg.ManifestFile> allManifestFiles(@NotNull @NotNull org.apache.iceberg.Table table, @NotNull @NotNull org.apache.iceberg.Snapshot snapshot) Get a stream of allManifestFile
objects from the givenTable
andSnapshot
.- Parameters:
table
- TheTable
to retrieve manifest files for.snapshot
- TheSnapshot
to retrieve manifest files from.- Returns:
- A stream of
ManifestFile
objects.
-
toStream
Convert aCloseableIterable
to aStream
that will close the iterable when the stream is closed. -
convertToDHType
Convert an Iceberg data type to a Deephaven type.- Parameters:
icebergType
- The Iceberg data type to be converted.- Returns:
- The converted Deephaven type.
-
convertToIcebergType
Convert a Deephaven type to an Iceberg type.- Parameters:
columnType
- The Deephaven type to be converted.- Returns:
- The converted Iceberg type.
-
createSpecAndSchema
public static IcebergUtils.SpecAndSchema createSpecAndSchema(@NotNull @NotNull TableDefinition tableDefinition) - Returns:
- A
IcebergUtils.SpecAndSchema
object containing the partition spec and schema, andnull
for read instructions.
-
createPartitionSpec
-
createNamespaceIfNotExists
public static boolean createNamespaceIfNotExists(@NotNull @NotNull org.apache.iceberg.catalog.Catalog catalog, @NotNull @NotNull org.apache.iceberg.catalog.Namespace namespace) -
dropNamespaceIfExists
public static boolean dropNamespaceIfExists(@NotNull @NotNull org.apache.iceberg.catalog.Catalog catalog, @NotNull @NotNull org.apache.iceberg.catalog.Namespace namespace) -
verifyRequiredFields
public static void verifyRequiredFields(org.apache.iceberg.Schema tableSchema, TableDefinition tableDefinition) Check that all required fields are present in the table definition -
verifyPartitioningColumns
public static void verifyPartitioningColumns(org.apache.iceberg.PartitionSpec tablePartitionSpec, TableDefinition tableDefinition) Check that all the partitioning columns from the partition spec are present in the Table Definition.
-