alluxio.client.block.stream

Class GrpcDataWriter

java.lang.Object

alluxio.client.block.stream.GrpcDataWriter

All Implemented Interfaces:

DataWriter, Cancelable, Closeable, AutoCloseable

@NotThreadSafe
public final class GrpcDataWriter
extends Object
implements DataWriter

A gRPC data writer that streams a full block or a UFS file to a gRPC data server. Protocol: 1. The client streams data chunks (start from pos 0) to the server. The client pauses if the client buffer is full, resumes if the buffer is not full. 2. The server reads chunks from the stream and writes them to the block worker. See the server side implementation for details. 3. The client can either complete or cancel the stream to end the write request. The client has to wait for the complete or cancel response from the data server to make sure that the server has cleaned its states. 4. To make it simple to handle errors, the stream is closed if any error occurs. NOTE: this class is NOT threadsafe. Do not call cancel/close while some other threads are writing.

Nested Class Summary

Nested classes/interfaces inherited from interface alluxio.client.block.stream.DataWriter

DataWriter.Factory

Method Summary

Modifier and Type	Method and Description
void	cancel() Cancels an operation.
int	chunkSize()
void	close()
static GrpcDataWriter	create(FileSystemContext context, WorkerNetAddress address, long id, long length, RequestType type, OutStreamOptions options)
void	flush() Flushes all the pending chunks.
Optional<String>	getUfsContentHash()
long	pos()
void	writeChunk(io.netty.buffer.ByteBuf buf) Writes a chunk.
void	writeFallbackInitRequest(long pos) Notifies the server UFS fallback endpoint to start writing a new block by resuming the given number of bytes from block store.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Method Detail

create

public static GrpcDataWriter create(FileSystemContext context,
                                    WorkerNetAddress address,
                                    long id,
                                    long length,
                                    RequestType type,
                                    OutStreamOptions options)
                             throws IOException

Parameters:

context - the file system context

address - the data server address

id - the block or UFS ID

length - the length of the block or file to write, set to Long.MAX_VALUE if unknown

type - type of the write request

options - the options of the output stream

Returns:

an instance of GrpcDataWriter

Throws:

IOException

pos

public long pos()

Specified by:

pos in interface DataWriter

Returns:

the current pos which is the same as the totally number of bytes written so far

getUfsContentHash

public Optional<String> getUfsContentHash()

Specified by:

getUfsContentHash in interface DataWriter

Returns:

the content hash of the file if it is written to the UFS. Will only return a non-empty value after the data writer has been closed.

writeChunk

public void writeChunk(io.netty.buffer.ByteBuf buf)
                throws IOException

Description copied from interface: DataWriter

Writes a chunk. This method takes the ownership of this chunk even if it fails to write the chunk.

Specified by:

writeChunk in interface DataWriter

Parameters:

buf - the chunk

Throws:

IOException

writeFallbackInitRequest

public void writeFallbackInitRequest(long pos)
                              throws IOException

Notifies the server UFS fallback endpoint to start writing a new block by resuming the given number of bytes from block store.

Parameters:

pos - number of bytes already written to block store

Throws:

IOException

cancel

public void cancel()

Description copied from interface: Cancelable

Cancels an operation.

Specified by:

cancel in interface Cancelable

flush

public void flush()
           throws IOException

Description copied from interface: DataWriter

Flushes all the pending chunks.

Specified by:

flush in interface DataWriter

Throws:

IOException

close

public void close()
           throws IOException

Specified by:

close in interface Closeable

Specified by:

close in interface AutoCloseable

Throws:

IOException

chunkSize

public int chunkSize()

Specified by:

chunkSize in interface DataWriter

Returns:

the chunk size in bytes used