alluxio.underfs.s3a

Class S3AInputStream

java.lang.Object

java.io.InputStream

alluxio.underfs.s3a.S3AInputStream

All Implemented Interfaces:

Positioned, Seekable, Closeable, AutoCloseable

@NotThreadSafe
public class S3AInputStream
extends InputStream
implements Seekable

An S3A input stream that supports skip and seek efficiently. Recommended wrap around an BufferedInputStream or SeekableBufferedInputStream to improve performance.

Field Summary

Fields

Modifier and Type	Field and Description
protected String	mBucketName Name of the bucket the object resides in.
protected com.amazonaws.services.s3.AmazonS3	mClient Client for operations with s3.
protected String	mKey The path of the object to read.
protected long	mPos The current position of the stream.
protected com.amazonaws.services.s3.model.GetObjectRequest	mReadRequest
protected RetryPolicy	mRetryPolicy Policy determining the retry behavior in case the key does not exist.
protected byte[]	mSingleByteHolder

Constructor Summary

Constructors

Constructor and Description
S3AInputStream(String bucketName, String key, com.amazonaws.services.s3.AmazonS3 client, long position, RetryPolicy retryPolicy) Constructor for an input stream of an object in s3 using the aws-sdk implementation to read the data.

Method Summary

Modifier and Type	Method and Description
protected com.amazonaws.services.s3.AmazonS3	getClient()
long	getPos()
int	read()
int	read(byte[] b)
int	read(byte[] b, int offset, int length)
void	seek(long pos) Moves the starting read position of the stream to the specified position which is relative to the start of the stream.
long	skip(long n)

Methods inherited from class java.io.InputStream

available, close, mark, markSupported, reset

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

mClient

protected com.amazonaws.services.s3.AmazonS3 mClient

Client for operations with s3.

mBucketName

protected final String mBucketName

Name of the bucket the object resides in.

mKey

protected final String mKey

The path of the object to read.

mSingleByteHolder

protected final byte[] mSingleByteHolder

mReadRequest

protected final com.amazonaws.services.s3.model.GetObjectRequest mReadRequest

mPos

protected long mPos

The current position of the stream.

mRetryPolicy

protected final RetryPolicy mRetryPolicy

Policy determining the retry behavior in case the key does not exist. The key may not exist because of eventual consistency.

Constructor Detail

S3AInputStream

public S3AInputStream(String bucketName,
                      String key,
                      com.amazonaws.services.s3.AmazonS3 client,
                      long position,
                      RetryPolicy retryPolicy)

Constructor for an input stream of an object in s3 using the aws-sdk implementation to read the data. The stream will be positioned at the specified position.

Parameters:

bucketName - the bucket the object resides in

key - the path of the object to read

client - the s3 client to use for operations

position - the position to begin reading from

retryPolicy - retry policy in case the key does not exist

Method Detail

read

public int read()
         throws IOException

Specified by:

read in class InputStream

Throws:

IOException

read

public int read(byte[] b)
         throws IOException

Overrides:

read in class InputStream

Throws:

IOException

read

public int read(byte[] b,
                int offset,
                int length)
         throws IOException

Overrides:

read in class InputStream

Throws:

IOException

skip

public long skip(long n)

Overrides:

skip in class InputStream

getPos

public long getPos()

Specified by:

getPos in interface Positioned

Returns:

the current position

seek

public void seek(long pos)

Description copied from interface: Seekable

Moves the starting read position of the stream to the specified position which is relative to the start of the stream. Seeking to a position before the current read position is supported.

Specified by:

seek in interface Seekable

Parameters:

pos - the position to seek to, it must be between 0 and the end of the stream - 1

getClient

protected com.amazonaws.services.s3.AmazonS3 getClient()

Returns:

the client