alluxio.client.block.stream

Class BlockInStream

java.lang.Object

java.io.InputStream

alluxio.client.block.stream.BlockInStream

All Implemented Interfaces:

BoundedStream, CanUnbuffer, PositionedReadable, Positioned, Seekable, Closeable, AutoCloseable

@NotThreadSafe
public class BlockInStream
extends InputStream
implements BoundedStream, Seekable, PositionedReadable, CanUnbuffer

Provides an InputStream implementation that is based on DataReaders to stream data chunk by chunk.

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static class	BlockInStream.BlockInStreamSource the source tracking where the block is from.

Field Summary

Fields

Modifier and Type	Field and Description
protected DataBuffer	mCurrentChunk The current data chunk.
protected DataReader	mDataReader

Constructor Summary

Constructors

Modifier	Constructor and Description
protected	BlockInStream(DataReader.Factory dataReaderFactory, WorkerNetAddress address, BlockInStream.BlockInStreamSource blockSource, long id, long length) Creates an instance of BlockInStream.

Method Summary

Modifier and Type	Method and Description
void	close()
static BlockInStream	create(FileSystemContext context, BlockInfo info, WorkerNetAddress dataSource, BlockInStream.BlockInStreamSource dataSourceType, InStreamOptions options) Creates a BlockInStream.
static BlockInStream	createRemoteBlockInStream(FileSystemContext context, long blockId, WorkerNetAddress address, BlockInStream.BlockInStreamSource blockSource, long blockSize, Protocol.OpenUfsBlockOptions ufsOptions) Creates a BlockInStream to read from a specific remote server.
WorkerNetAddress	getAddress()
DataReader.Factory	getDataReaderFactory()
long	getId()
long	getPos()
BlockInStream.BlockInStreamSource	getSource()
int	positionedRead(long pos, byte[] b, int off, int len) Reads up to the specified number of bytes, from a given position within a file, and return the number of bytes read.
int	read()
int	read(byte[] b)
int	read(byte[] b, int off, int len)
int	read(ByteBuffer byteBuffer, int off, int len) Reads up to len bytes of data from the input stream into the byte buffer.
long	remaining()
void	seek(long pos) Moves the starting read position of the stream to the specified position which is relative to the start of the stream.
long	skip(long n)
void	unbuffer() Reduce the buffering.

Methods inherited from class java.io.InputStream

available, mark, markSupported, reset

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

mCurrentChunk

protected DataBuffer mCurrentChunk

The current data chunk.

mDataReader

protected DataReader mDataReader

Constructor Detail

BlockInStream

protected BlockInStream(DataReader.Factory dataReaderFactory,
                        WorkerNetAddress address,
                        BlockInStream.BlockInStreamSource blockSource,
                        long id,
                        long length)

Creates an instance of BlockInStream.

Parameters:

dataReaderFactory - the data reader factory

address - the address of the gRPC data server

blockSource - the source location of the block

id - the ID (either block ID or UFS file ID)

length - the length

Method Detail

create

public static BlockInStream create(FileSystemContext context,
                                   BlockInfo info,
                                   WorkerNetAddress dataSource,
                                   BlockInStream.BlockInStreamSource dataSourceType,
                                   InStreamOptions options)
                            throws IOException

Creates a BlockInStream. One of several read behaviors: 1. Domain socket - if the data source is the local worker and the local worker has a domain socket server 2. Short-Circuit - if the data source is the local worker 3. Local Loopback Read - if the data source is the local worker and short circuit is disabled 4. Read from remote worker - if the data source is a remote worker 5. UFS Read from worker - if the data source is UFS, read from the UFS policy's designated worker (ufs -> local or remote worker -> client)

Parameters:

context - the file system context

info - the block info

dataSource - the Alluxio worker which should read the data

dataSourceType - the source location of the block

options - the InStream options

Returns:

the BlockInStream object

Throws:

IOException

createRemoteBlockInStream

public static BlockInStream createRemoteBlockInStream(FileSystemContext context,
                                                      long blockId,
                                                      WorkerNetAddress address,
                                                      BlockInStream.BlockInStreamSource blockSource,
                                                      long blockSize,
                                                      Protocol.OpenUfsBlockOptions ufsOptions)

Creates a BlockInStream to read from a specific remote server. Should only be used in cases where the data source and method of reading is known, i.e. worker - worker communication.

Parameters:

context - the file system context

blockId - the block id

address - the address of the gRPC data server

blockSource - the source location of the block

blockSize - the size of the block

ufsOptions - the ufs read options

Returns:

the BlockInStream created

getPos

public long getPos()

Specified by:

getPos in interface Positioned

Returns:

the current position

read

public int read()
         throws IOException

Specified by:

read in class InputStream

Throws:

IOException

read

public int read(byte[] b)
         throws IOException

Overrides:

read in class InputStream

Throws:

IOException

read

public int read(byte[] b,
                int off,
                int len)
         throws IOException

Overrides:

read in class InputStream

Throws:

IOException

read

public int read(ByteBuffer byteBuffer,
                int off,
                int len)
         throws IOException

Reads up to len bytes of data from the input stream into the byte buffer.

Parameters:

byteBuffer - the buffer into which the data is read

off - the start offset in the buffer at which the data is written

len - the maximum number of bytes to read

Returns:

the total number of bytes read into the buffer, or -1 if there is no more data because the end of the stream has been reached

Throws:

IOException

positionedRead

public int positionedRead(long pos,
                          byte[] b,
                          int off,
                          int len)
                   throws IOException

Description copied from interface: PositionedReadable

Reads up to the specified number of bytes, from a given position within a file, and return the number of bytes read. This does not change the current offset of a file, and is thread-safe. Not all implementations meet the thread safety requirement. According to the documentation of the PositionedReadable interface in Hadoop, those that do not meet the requirement cannot be used as a backing store for some applications, such as Apache HBase. All implementations of this interface in Alluxio must meet the thread safety requirement.

Specified by:

positionedRead in interface PositionedReadable

Parameters:

pos - position within file

b - destination buffer

off - offset in the buffer

len - number of bytes to read

Returns:

actual number of bytes read; -1 means "EOF";

Throws:

IOException

remaining

public long remaining()

Specified by:

remaining in interface BoundedStream

Returns:

the remaining number of bytes left in the stream, starting at the current position

seek

public void seek(long pos)
          throws IOException

Description copied from interface: Seekable

Moves the starting read position of the stream to the specified position which is relative to the start of the stream. Seeking to a position before the current read position is supported.

Specified by:

seek in interface Seekable

Parameters:

pos - the position to seek to, it must be between 0 and the end of the stream - 1

Throws:

IOException

skip

public long skip(long n)
          throws IOException

Overrides:

skip in class InputStream

Throws:

IOException

close

public void close()
           throws IOException

Specified by:

close in interface Closeable

Specified by:

close in interface AutoCloseable

Overrides:

close in class InputStream

Throws:

IOException

getDataReaderFactory

public DataReader.Factory getDataReaderFactory()

Returns:

the underlying data reader factory

unbuffer

public void unbuffer()

Description copied from interface: CanUnbuffer

Reduce the buffering. This will also free sockets and file descriptors held by the stream, if possible.

Specified by:

unbuffer in interface CanUnbuffer

getAddress

public WorkerNetAddress getAddress()

Returns:

the address of the data server

getSource

public BlockInStream.BlockInStreamSource getSource()

Returns:

the source of the block location

getId

public long getId()

Returns:

the block ID