alluxio.client.block.stream

Class BlockInStream

java.lang.Object

java.io.InputStream

alluxio.client.block.stream.BlockInStream

All Implemented Interfaces:

BoundedStream, PositionedReadable, Positioned, Seekable, Closeable, AutoCloseable

@NotThreadSafe
public class BlockInStream
extends InputStream
implements BoundedStream, Seekable, PositionedReadable

Provides an InputStream implementation that is based on DataReaders to stream data chunk by chunk.

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static class	BlockInStream.BlockInStreamSource the source tracking where the block is from.

Constructor Summary

Constructors

Modifier	Constructor and Description
protected	BlockInStream(DataReader.Factory dataReaderFactory, AlluxioConfiguration conf, WorkerNetAddress address, BlockInStream.BlockInStreamSource blockSource, long id, long length) Creates an instance of BlockInStream.

Method Summary

Modifier and Type	Method and Description
void	close()
static BlockInStream	create(FileSystemContext context, BlockInfo info, WorkerNetAddress dataSource, BlockInStream.BlockInStreamSource dataSourceType, InStreamOptions options) Creates a BlockInStream.
static BlockInStream	createRemoteBlockInStream(FileSystemContext context, long blockId, WorkerNetAddress address, BlockInStream.BlockInStreamSource blockSource, long blockSize, Protocol.OpenUfsBlockOptions ufsOptions) Creates a BlockInStream to read from a specific remote server.
WorkerNetAddress	getAddress()
DataReader.Factory	getDataReaderFactory()
long	getId()
long	getPos()
BlockInStream.BlockInStreamSource	getSource()
int	positionedRead(long pos, byte[] b, int off, int len) Reads up to the specified number of bytes, from a given position within a file, and return the number of bytes read.
int	read()
int	read(byte[] b)
int	read(byte[] b, int off, int len)
int	read(ByteBuffer byteBuffer, int off, int len) Reads up to len bytes of data from the input stream into the byte buffer.
long	remaining()
void	seek(long pos) Moves the starting read position of the stream to the specified position which is relative to the start of the stream.
long	skip(long n)

Methods inherited from class java.io.InputStream

available, mark, markSupported, reset

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

BlockInStream

protected BlockInStream(DataReader.Factory dataReaderFactory,
                        AlluxioConfiguration conf,
                        WorkerNetAddress address,
                        BlockInStream.BlockInStreamSource blockSource,
                        long id,
                        long length)

Creates an instance of BlockInStream.

Parameters:

dataReaderFactory - the data reader factory

conf - the Alluxio configuration

address - the address of the gRPC data server

blockSource - the source location of the block

id - the ID (either block ID or UFS file ID)

length - the length

Method Detail

create

public static BlockInStream create(FileSystemContext context,
                                   BlockInfo info,
                                   WorkerNetAddress dataSource,
                                   BlockInStream.BlockInStreamSource dataSourceType,
                                   InStreamOptions options)
                            throws IOException

Creates a BlockInStream. One of several read behaviors: 1. Domain socket - if the data source is the local worker and the local worker has a domain socket server 2. Short-Circuit - if the data source is the local worker 3. Local Loopback Read - if the data source is the local worker and short circuit is disabled 4. Read from remote worker - if the data source is a remote worker 5. UFS Read from worker - if the data source is UFS, read from the UFS policy's designated worker (ufs -> local or remote worker -> client)

Parameters:

context - the file system context

info - the block info

dataSource - the Alluxio worker which should read the data

dataSourceType - the source location of the block

options - the instream options

Returns:

the BlockInStream object

Throws:

IOException

createRemoteBlockInStream

public static BlockInStream createRemoteBlockInStream(FileSystemContext context,
                                                      long blockId,
                                                      WorkerNetAddress address,
                                                      BlockInStream.BlockInStreamSource blockSource,
                                                      long blockSize,
                                                      Protocol.OpenUfsBlockOptions ufsOptions)

Creates a BlockInStream to read from a specific remote server. Should only be used in cases where the data source and method of reading is known, ie. worker - worker communication.

Parameters:

context - the file system context

blockId - the block id

address - the address of the gRPC data server

blockSource - the source location of the block

blockSize - the size of the block

ufsOptions - the ufs read options

Returns:

the BlockInStream created

getPos

public long getPos()

Specified by:

getPos in interface Positioned

Returns:

the current position

read

public int read()
         throws IOException

Specified by:

read in class InputStream

Throws:

IOException

read

public int read(byte[] b)
         throws IOException

Overrides:

read in class InputStream

Throws:

IOException

read

public int read(byte[] b,
                int off,
                int len)
         throws IOException

Overrides:

read in class InputStream

Throws:

IOException

read

public int read(ByteBuffer byteBuffer,
                int off,
                int len)
         throws IOException

Reads up to len bytes of data from the input stream into the byte buffer.

Parameters:

byteBuffer - the buffer into which the data is read

off - the start offset in the buffer at which the data is written

len - the maximum number of bytes to read

Returns:

the total number of bytes read into the buffer, or -1 if there is no more data because the end of the stream has been reached

Throws:

IOException

positionedRead

public int positionedRead(long pos,
                          byte[] b,
                          int off,
                          int len)
                   throws IOException

Description copied from interface: PositionedReadable

Reads up to the specified number of bytes, from a given position within a file, and return the number of bytes read. This does not change the current offset of a file, and is thread-safe. Not all implementations meet the thread safety requirement. According to the documentation of the PositionedReadable interface in Hadoop, those that do not meet the requirement cannot be used as a backing store for some applications, such as Apache HBase. All implementations of this interface in Alluxio must meet the thread safety requirement.

Specified by:

positionedRead in interface PositionedReadable

Parameters:

pos - position within file

b - destination buffer

off - offset in the buffer

len - number of bytes to read

Returns:

actual number of bytes read; -1 means "EOF";

Throws:

IOException

remaining

public long remaining()

Specified by:

remaining in interface BoundedStream

Returns:

the remaining number of bytes left in the stream, starting at the current position

seek

public void seek(long pos)
          throws IOException

Description copied from interface: Seekable

Moves the starting read position of the stream to the specified position which is relative to the start of the stream. Seeking to a position before the current read position is supported.

Specified by:

seek in interface Seekable

Parameters:

pos - the position to seek to, it must be between 0 and the end of the stream - 1

Throws:

IOException

skip

public long skip(long n)
          throws IOException

Overrides:

skip in class InputStream

Throws:

IOException

close

public void close()
           throws IOException

Specified by:

close in interface Closeable

Specified by:

close in interface AutoCloseable

Overrides:

close in class InputStream

Throws:

IOException

getDataReaderFactory

public DataReader.Factory getDataReaderFactory()

Returns:

the underlying data reader factory

getAddress

public WorkerNetAddress getAddress()

Returns:

the address of the data server

getSource

public BlockInStream.BlockInStreamSource getSource()

Returns:

the source of the block location

getId

public long getId()

Returns:

the block ID