Package io.deephaven.parquet.compress

Interface CompressorAdapter

All Superinterfaces:

AutoCloseable, SafeCloseable

public interface CompressorAdapter
extends SafeCloseable

An Intermediate adapter interface between Deephaven column writing and parquet compression.

Nested Class Summary

Nested Classes

Modifier and Type	Interface	Description
static final class	CompressorAdapter.InputStreamNoClose	An InputStream that will not close the underlying stream when InputStream.close() is called.
static interface	CompressorAdapter.ResourceCache

Field Summary

Fields

Modifier and Type	Field	Description
static final CompressorAdapter	PASSTHRU	An CompressorAdapter instance that reads and writes uncompressed data directly.

Method Summary

Modifier and Type	Method	Description
OutputStream	compress(OutputStream os)	Creates a new output stream that will take uncompressed writes, and flush data to the provided stream as compressed data.
InputStream	decompress(InputStream inputStream, int compressedSize, int uncompressedSize, CompressorAdapter.ResourceCache decompressorCache)	Creates a new input stream that will read compressed data from the provided stream, and return uncompressed data.
org.apache.parquet.hadoop.metadata.CompressionCodecName	getCodecName()
static void	readNBytes(InputStream in, byte[] bytes, int offset, int length)	Reads exactly length number of bytes from the input stream into the provided byte array.
static byte[]	readNBytes(InputStream in, int length)	Reads exactly length number of bytes from the input stream into a new byte array and returns it.
void	reset()	Reset the internal state of this CompressorAdapter so more rows can be read or written.

Methods inherited from interface io.deephaven.util.SafeCloseable

close

Field Details

PASSTHRU

static final CompressorAdapter PASSTHRU

An CompressorAdapter instance that reads and writes uncompressed data directly.

Method Details

readNBytes

static void readNBytes(InputStream in,
 byte[] bytes,
 int offset,
 int length)
                throws IOException

Reads exactly length number of bytes from the input stream into the provided byte array.

Parameters:

in - The input stream to read from

bytes - The byte array to read into

offset - The offset in the byte array to start reading at

length - The number of bytes to read

Throws:

IOException - If an error occurs while reading from the input stream

IllegalArgumentException - If the byte array is too small to read length number of bytes

IOException - If the expected number of bytes could not be read

readNBytes

static byte[] readNBytes(InputStream in,
 int length)
                  throws IOException

Reads exactly length number of bytes from the input stream into a new byte array and returns it.

Throws:

IOException

See Also:

• readNBytesHelper(InputStream, byte[], int, int)

compress

OutputStream compress(OutputStream os)
               throws IOException

Creates a new output stream that will take uncompressed writes, and flush data to the provided stream as compressed data.

Note that this method is not thread safe.

Parameters:

os - the output stream to write compressed contents to

Returns:

an output stream that can accept writes

Throws:

IOException - thrown if an error occurs writing data

decompress

InputStream decompress(InputStream inputStream,
 int compressedSize,
 int uncompressedSize,
 CompressorAdapter.ResourceCache decompressorCache)
                throws IOException

Creates a new input stream that will read compressed data from the provided stream, and return uncompressed data. Caller should not close the returned InputStream because this might return an internally cached decompressor to the pool. Also, callers should not read more data from the returned InputStream after closing the argument inputStream.

The provided DecompressorHolder is used for decompressing if compatible with the compression codec. Otherwise, a new decompressor is created and set in the DecompressorHolder.

Note that this method is thread safe, assuming the cached decompressor instances are not shared across threads.

Parameters:

inputStream - an input stream containing compressed data

compressedSize - the number of bytes in the compressed data

uncompressedSize - the number of bytes that should be present when decompressed

decompressorCache - Used to cache Decompressor instances for reuse

Returns:

an input stream that will return uncompressed data

Throws:

IOException - thrown if an error occurs reading data.

getCodecName

org.apache.parquet.hadoop.metadata.CompressionCodecName getCodecName()

Returns:

the CompressionCodecName enum value that represents this compressor.

reset

void reset()

Reset the internal state of this CompressorAdapter so more rows can be read or written.

This method can be called after compress(java.io.OutputStream) to reset the internal state of the compressor, and is not required before compress(java.io.OutputStream), or before and after decompress(java.io.InputStream, int, int, io.deephaven.parquet.compress.CompressorAdapter.ResourceCache) because those methods internally manage their own state.