Package io.deephaven.extensions.s3

Class S3Instructions

java.lang.Object
io.deephaven.extensions.s3.S3Instructions
All Implemented Interfaces:
LogOutputAppendable
@Immutable public abstract class S3Instructions extends Object implements LogOutputAppendable
This class provides instructions intended for reading from and writing to S3-compatible APIs. The default values documented in this class may change in the future. As such, callers may wish to explicitly set the values.

  • Constructor Details

    • S3Instructions

      public S3Instructions()

  • Method Details

    • builder

      public static S3Instructions.Builder builder()

    • regionName

      public abstract Optional<String> regionName()
      The region name to use when reading or writing to S3. If not provided, the region name is picked by the AWS SDK from 'aws.region' system property, "AWS_REGION" environment variable, the {user.home}/.aws/credentials or {user.home}/.aws/config files, or from EC2 metadata service, if running in EC2.

    • maxConcurrentRequests

      @Default public int maxConcurrentRequests()
      The maximum number of concurrent requests to make to S3, defaults to 256.

    • readAheadCount

      @Default public int readAheadCount()
      The number of fragments to send asynchronous read requests for while reading the current fragment. Defaults to 32, which means by default, we will fetch 32 fragments in advance when reading current fragment.

    • fragmentSize

      @Default public int fragmentSize()
      The maximum byte size of each fragment to read from S3, defaults to 65536, must be larger than 8192. If there are fewer bytes remaining in the file, the fetched fragment can be smaller.

    • connectionTimeout

      @Default public Duration connectionTimeout()
      The amount of time to wait when initially establishing a connection before giving up and timing out, defaults to 2 seconds.

    • readTimeout

      @Default public Duration readTimeout()
      The amount of time to wait when reading a fragment before giving up and timing out, defaults to 2 seconds. The implementation may choose to internally retry the request multiple times, so long as the total time does not exceed this timeout.

    • credentials

      @Default public Credentials credentials()
      The credentials to use when reading or writing to S3. By default, uses Credentials.defaultCredentials().

    • writePartSize

      @Default public int writePartSize()
      The size of each part (in bytes) to upload when writing to S3, defaults to 10485760. The minimum allowed part size is 5242880. Setting a higher value may increase throughput, but may also increase memory usage. Note that the maximum number of parts allowed for a single file is 10,000. Therefore, for 10485760 part size, the maximum size of a single file that can be written is 10485760 * 10,000 bytes.

    • numConcurrentWriteParts

      @Default public int numConcurrentWriteParts()
      The maximum number of parts that can be uploaded concurrently when writing to S3 without blocking. Setting a higher value may increase throughput, but may also increase memory usage. Defaults to 64.

    • append

      public LogOutput append(LogOutput logOutput)
      Specified by:
      append in interface LogOutputAppendable

    • endpointOverride

      public abstract Optional<URI> endpointOverride()
      The endpoint to connect to. Callers connecting to AWS do not typically need to set this; it is most useful when connecting to non-AWS, S3-compatible APIs.
      See Also: