Package io.deephaven.iceberg.layout

Class IcebergBaseLayout

java.lang.Object
io.deephaven.iceberg.layout.IcebergBaseLayout
All Implemented Interfaces:
TableLocationKeyFinder<IcebergTableLocationKey>
Direct Known Subclasses:
IcebergFlatLayout, IcebergKeyValuePartitionedLayout
public abstract class IcebergBaseLayout extends Object implements TableLocationKeyFinder<IcebergTableLocationKey>

  • Constructor Details

    • IcebergBaseLayout

      public IcebergBaseLayout(@NotNull @NotNull IcebergTableAdapter tableAdapter, @NotNull @NotNull IcebergReadInstructions instructions, @NotNull @NotNull DataInstructionsProviderLoader dataInstructionsProvider)
      Parameters:
      tableAdapter - The IcebergTableAdapter that will be used to access the table.
      instructions - The instructions for customizations while reading.
      dataInstructionsProvider - The provider for special instructions, to be used if special instructions not provided in the instructions.

  • Method Details

    • locationKey

      protected IcebergTableLocationKey locationKey(@NotNull @NotNull org.apache.iceberg.ManifestFile manifestFile, @NotNull @NotNull org.apache.iceberg.DataFile dataFile, @NotNull @NotNull URI fileUri, @Nullable @Nullable Map<String,Comparable<?>> partitions, @NotNull @NotNull SeekableChannelsProvider channelsProvider)
      Create a new IcebergTableLocationKey for the given ManifestFile, DataFile and URI.
      Parameters:
      manifestFile - The manifest file from which the data file was discovered
      dataFile - The data file that backs the keyed location
      fileUri - The URI for the file that backs the keyed location
      partitions - The table partitions enclosing the table location keyed by the returned key. If null, the location will be a member of no partitions.
      Returns:
      A new IcebergTableLocationKey

    • findKeys

      public void findKeys(@NotNull @NotNull Consumer<IcebergTableLocationKey> locationKeyObserver)
      Description copied from interface: TableLocationKeyFinder
      Find keys and deliver them to the locationKeyObserver.
      Specified by:
      findKeys in interface TableLocationKeyFinder<IcebergTableLocationKey>
      Parameters:
      locationKeyObserver - Per-key callback

    • maybeUpdateSnapshot

      protected boolean maybeUpdateSnapshot()
      Update the snapshot to the latest snapshot from the catalog if

    • updateSnapshot

      protected void updateSnapshot(long snapshotId)
      Update the snapshot to the user specified snapshot. See IcebergTable.update(long) for more details.

    • updateSnapshot

      protected void updateSnapshot(@NotNull @NotNull org.apache.iceberg.Snapshot updateSnapshot)
      Update the snapshot to the user specified snapshot. See IcebergTable.update(Snapshot) for more details.

    • computeSortedColumns

      @VisibleForTesting @NotNull public static @NotNull List<SortColumn> computeSortedColumns(@NotNull org.apache.iceberg.Table icebergTable, @NotNull @NotNull org.apache.iceberg.DataFile dataFile, @NotNull @NotNull ParquetInstructions readInstructions)