alluxio.underfs.s3a

Class S3AInputStream

java.lang.Object

java.io.InputStream

alluxio.underfs.s3a.S3AInputStream

All Implemented Interfaces:

Closeable, AutoCloseable

@NotThreadSafe
public class S3AInputStream
extends InputStream

A wrapper around an S3ObjectInputStream which handles skips efficiently.

Field Summary

Fields

Modifier and Type	Field and Description
protected String	mBucketName Name of the bucket the object resides in.
protected com.amazonaws.services.s3.AmazonS3	mClient Client for operations with s3.
protected com.amazonaws.services.s3.model.S3ObjectInputStream	mIn The backing input stream from s3.
protected String	mKey The path of the object to read.
protected long	mPos The current position of the stream.
protected RetryPolicy	mRetryPolicy Policy determining the retry behavior in case the key does not exist.

Constructor Summary

Constructors

Constructor and Description
S3AInputStream(String bucketName, String key, com.amazonaws.services.s3.AmazonS3 client, long position, RetryPolicy retryPolicy) Constructor for an input stream of an object in s3 using the aws-sdk implementation to read the data.
S3AInputStream(String bucketName, String key, com.amazonaws.services.s3.AmazonS3 client, RetryPolicy retryPolicy) Constructor for an input stream of an object in s3 using the aws-sdk implementation to read the data.

Method Summary

Modifier and Type	Method and Description
void	close()
protected com.amazonaws.services.s3.AmazonS3	getClient()
int	read()
int	read(byte[] b)
int	read(byte[] b, int offset, int length)
long	skip(long n)

Methods inherited from class java.io.InputStream

available, mark, markSupported, reset

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

mClient

protected com.amazonaws.services.s3.AmazonS3 mClient

Client for operations with s3.

mBucketName

protected final String mBucketName

Name of the bucket the object resides in.

mKey

protected final String mKey

The path of the object to read.

mIn

protected com.amazonaws.services.s3.model.S3ObjectInputStream mIn

The backing input stream from s3.

mPos

protected long mPos

The current position of the stream.

mRetryPolicy

protected final RetryPolicy mRetryPolicy

Policy determining the retry behavior in case the key does not exist. The key may not exist because of eventual consistency.

Constructor Detail

S3AInputStream

public S3AInputStream(String bucketName,
                      String key,
                      com.amazonaws.services.s3.AmazonS3 client,
                      RetryPolicy retryPolicy)

Constructor for an input stream of an object in s3 using the aws-sdk implementation to read the data. The stream will be positioned at the start of the file.

Parameters:

bucketName - the bucket the object resides in

key - the path of the object to read

client - the s3 client to use for operations

retryPolicy - retry policy in case the key does not exist

S3AInputStream

public S3AInputStream(String bucketName,
                      String key,
                      com.amazonaws.services.s3.AmazonS3 client,
                      long position,
                      RetryPolicy retryPolicy)

Constructor for an input stream of an object in s3 using the aws-sdk implementation to read the data. The stream will be positioned at the specified position.

Parameters:

bucketName - the bucket the object resides in

key - the path of the object to read

client - the s3 client to use for operations

position - the position to begin reading from

retryPolicy - retry policy in case the key does not exist

Method Detail

close

public void close()

Specified by:

close in interface Closeable

Specified by:

close in interface AutoCloseable

Overrides:

close in class InputStream

read

public int read()
         throws IOException

Specified by:

read in class InputStream

Throws:

IOException

read

public int read(byte[] b)
         throws IOException

Overrides:

read in class InputStream

Throws:

IOException

read

public int read(byte[] b,
                int offset,
                int length)
         throws IOException

Overrides:

read in class InputStream

Throws:

IOException

skip

public long skip(long n)
          throws IOException

Overrides:

skip in class InputStream

Throws:

IOException

getClient

protected com.amazonaws.services.s3.AmazonS3 getClient()

Returns:

the client