alluxio.client.file.dora.netty

Class NettyDataWriter

java.lang.Object

alluxio.client.file.dora.netty.NettyDataWriter

All Implemented Interfaces:

DataWriter, Cancelable, Closeable, AutoCloseable

@NotThreadSafe
public class NettyDataWriter
extends Object
implements DataWriter

A netty packet writer that streams a full block or a UFS file to a netty data server.

Protocol: 1. The client streams packets (start from pos 0) to the server. The client pauses if the client buffer is full, resumes if the buffer is not full. 2. The server reads packets from the channel and writes them to the block worker. See the server side implementation for details. 3. The client can either send an EOF packet or a CANCEL packet to end the write request. The client has to wait for the response from the data server for the EOF or CANCEL packet to make sure that the server has cleaned its states. 4. To make it simple to handle errors, the channel is closed if any error occurs.

NOTE: this class is NOT threadsafe. Do not call cancel/close while some other threads are writing.

Nested Class Summary

Nested classes/interfaces inherited from interface alluxio.client.block.stream.DataWriter

DataWriter.Factory

Field Summary

Fields

Modifier and Type	Field and Description
protected Protocol.WriteRequest	mPartialRequest

Constructor Summary

Constructors

Modifier	Constructor and Description
protected	NettyDataWriter(FileSystemContext context, WorkerNetAddress address, long length, long packetSize, RequestType type, OutStreamOptions options, io.netty.channel.Channel channel) Creates an instance of NettyDataWriter.

Method Summary

Modifier and Type	Method and Description
void	cancel() Cancels an operation.
int	chunkSize()
void	close()
static NettyDataWriter	create(FileSystemContext context, WorkerNetAddress address, long length, RequestType type, OutStreamOptions options)
void	flush() Flushes all the pending chunks.
Throwable	getPacketWriteException() Get the exception object that created during packet writing.
protected Protocol.RequestType	getRequestType(RequestType requestType)
Optional<String>	getUfsContentHash()
long	pos()
void	writeChunk(byte[] bytes, int off, int len) Write chunk by sending a write request.
void	writeChunk(io.netty.buffer.ByteBuf buf) Writes a chunk.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

mPartialRequest

protected Protocol.WriteRequest mPartialRequest

Constructor Detail

NettyDataWriter

protected NettyDataWriter(FileSystemContext context,
                          WorkerNetAddress address,
                          long length,
                          long packetSize,
                          RequestType type,
                          OutStreamOptions options,
                          io.netty.channel.Channel channel)

Creates an instance of NettyDataWriter.

Parameters:

context - the file system context

address - the data server address

length - the length of the file to write, set to Long.MAX_VALUE if unknown

packetSize - the packet size

type - type of the write request

options - details of the write request which are constant for all requests

channel - netty channel

Method Detail

create

public static NettyDataWriter create(FileSystemContext context,
                                     WorkerNetAddress address,
                                     long length,
                                     RequestType type,
                                     OutStreamOptions options)
                              throws IOException

Parameters:

context - the file system context

address - the data server address

length - the length of the block or file to write, set to Long.MAX_VALUE if unknown

type - type of the write request

options - the options of the output stream

Returns:

an instance of NettyDataWriter

Throws:

IOException

getRequestType

protected Protocol.RequestType getRequestType(RequestType requestType)

pos

public long pos()

Specified by:

pos in interface DataWriter

Returns:

the current pos which is the same as the totally number of bytes written so far

getUfsContentHash

public Optional<String> getUfsContentHash()

Specified by:

getUfsContentHash in interface DataWriter

Returns:

the content hash of the file if it is written to the UFS. Will only return a non-empty value after the data writer has been closed.

writeChunk

public void writeChunk(byte[] bytes,
                       int off,
                       int len)
                throws IOException

Write chunk by sending a write request.

Parameters:

bytes - the bytes to be written

off - the offset of the given byte array

len - the length to be written

Throws:

IOException

writeChunk

public void writeChunk(io.netty.buffer.ByteBuf buf)
                throws IOException

Description copied from interface: DataWriter

Writes a chunk. This method takes the ownership of this chunk even if it fails to write the chunk.

Specified by:

writeChunk in interface DataWriter

Parameters:

buf - the chunk

Throws:

IOException

cancel

public void cancel()

Description copied from interface: Cancelable

Cancels an operation.

Specified by:

cancel in interface Cancelable

flush

public void flush()
           throws IOException

Description copied from interface: DataWriter

Flushes all the pending chunks.

Specified by:

flush in interface DataWriter

Throws:

IOException

close

public void close()
           throws IOException

Specified by:

close in interface Closeable

Specified by:

close in interface AutoCloseable

Throws:

IOException

chunkSize

public int chunkSize()

Specified by:

chunkSize in interface DataWriter

Returns:

the chunk size in bytes used

getPacketWriteException

public Throwable getPacketWriteException()

Get the exception object that created during packet writing.

Returns:

the exception object