Package io.deephaven.extensions.s3

Class S3Instructions

java.lang.Object
io.deephaven.extensions.s3.S3Instructions
All Implemented Interfaces:
LogOutputAppendable
@Immutable public abstract class S3Instructions extends Object implements LogOutputAppendable
This class provides instructions intended for reading from and writing to S3-compatible APIs. The default values documented in this class may change in the future. As such, callers may wish to explicitly set the values.

  • Constructor Details

    • S3Instructions

      public S3Instructions()

  • Method Details

    • builder

      public static S3Instructions.Builder builder()

    • regionName

      public abstract String regionName()
      The region name to use when reading or writing to S3.

    • maxConcurrentRequests

      @Default public int maxConcurrentRequests()
      The maximum number of concurrent requests to make to S3, defaults to 50.

    • readAheadCount

      @Default public int readAheadCount()
      The number of fragments to send asynchronous read requests for while reading the current fragment. Defaults to 1, which means by default, we will fetch 1 fragments in advance when reading current fragment.

    • fragmentSize

      @Default public int fragmentSize()
      The maximum byte size of each fragment to read from S3, defaults to the value of config parameter "S3.maxFragmentSize", or 5 MiB if unset. Must be between 8 KiB and the value of config parameter "S3.maxFragmentSize". If there are fewer bytes remaining in the file, the fetched fragment can be smaller.

    • maxCacheSize

      @Default public int maxCacheSize()
      The maximum number of fragments to cache in memory, defaults to Math.max(1 + readAheadCount(), DEFAULT_MAX_CACHE_SIZE), which is at least 32. This caching is done at the deephaven layer for faster access to recently read fragments. Must be greater than or equal to 1 + readAheadCount().

    • connectionTimeout

      @Default public Duration connectionTimeout()
      The amount of time to wait when initially establishing a connection before giving up and timing out, defaults to 2 seconds.

    • readTimeout

      @Default public Duration readTimeout()
      The amount of time to wait when reading a fragment before giving up and timing out, defaults to 2 seconds. The implementation may choose to internally retry the request multiple times, so long as the total time does not exceed this timeout.

    • credentials

      @Default public Credentials credentials()
      The credentials to use when reading or writing to S3. By default, uses Credentials.defaultCredentials().

    • append

      public LogOutput append(LogOutput logOutput)
      Specified by:
      append in interface LogOutputAppendable

    • endpointOverride

      public abstract Optional<URI> endpointOverride()
      The endpoint to connect to. Callers connecting to AWS do not typically need to set this; it is most useful when connecting to non-AWS, S3-compatible APIs.
      See Also: