alluxio.client.file.dora

Class DoraCacheClient

java.lang.Object

alluxio.client.file.dora.DoraCacheClient

public class DoraCacheClient
extends Object

Dora cache client.

Field Summary

Fields

Modifier and Type	Field and Description
static int	DUMMY_BLOCK_ID

Constructor Summary

Constructors

Constructor and Description
DoraCacheClient(FileSystemContext context) Constructor.

Method Summary

Modifier and Type	Method and Description
Map<String,List<WorkerNetAddress>>	checkFileLocation(String path, GetStatusPOptions options) Get a map that maps file path to the workers list.
protected BlockWorkerInfo	choosePreferredWorker(List<BlockWorkerInfo> workers) Chooses a client preferred worker from multiple workers which hold multiple replicas.
void	completeFile(String path, CompleteFilePOptions options, String uuid) Mark the newly created and written file as complete.
void	createDirectory(String path, CreateDirectoryPOptions options) Create a dir.
Pair<URIStatus,String>	createFile(String path, CreateFilePOptions options) Create File.
protected GrpcDataReader.Factory	createGrpcDataReader(WorkerNetAddress workerNetAddress, Protocol.OpenUfsBlockOptions ufsOptions)
protected NettyDataReader	createNettyDataReader(WorkerNetAddress workerNetAddress, Protocol.OpenUfsBlockOptions ufsOptions)
DoraCachePositionReader	createNettyPositionReader(URIStatus status, Protocol.OpenUfsBlockOptions ufsOptions, Optional<CloseableSupplier<PositionReader>> externalPositionReader)
void	delete(String path, DeletePOptions options) Delete a file.
boolean	exists(String path, ExistsPOptions options) Check existence of a file or dir.
protected long	getChunkSize()
FileSystemContext	getContext() Get Context.
PositionReadFileInStream	getInStream(URIStatus status, Protocol.OpenUfsBlockOptions ufsOptions) Get a stream to read the data from dora cache cluster.
DoraFileOutStream	getOutStream(AlluxioURI alluxioPath, FileSystemContext fsContext, OutStreamOptions outStreamOptions, FileOutStream ufsOutStream, String uuid) Get a stream to write the data to dora cache cluster.
URIStatus	getStatus(String path, GetStatusPOptions options) Get status.
protected URIStatus	getStatusByGrpc(String path, GetStatusPOptions options)
WorkerNetAddress	getWorkerNetAddress(String path) Get the worker net address of the specific file path.
List<URIStatus>	listStatus(String path, ListStatusPOptions options) List Status from Worker.
void	rename(String src, String dst, RenamePOptions options) Rename a src file/dir to dst file/dir.
void	setAttribute(String path, SetAttributePOptions options) Set attributes for a file or dir.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

DUMMY_BLOCK_ID

public static final int DUMMY_BLOCK_ID

See Also:

Constant Field Values

Constructor Detail

DoraCacheClient

public DoraCacheClient(FileSystemContext context)

Constructor.

Parameters:

context -

Method Detail

getInStream

public PositionReadFileInStream getInStream(URIStatus status,
                                            Protocol.OpenUfsBlockOptions ufsOptions)

Get a stream to read the data from dora cache cluster.

Parameters:

status -

ufsOptions -

Returns:

the input stream

getOutStream

public DoraFileOutStream getOutStream(AlluxioURI alluxioPath,
                                      FileSystemContext fsContext,
                                      OutStreamOptions outStreamOptions,
                                      @Nullable
                                      FileOutStream ufsOutStream,
                                      String uuid)
                               throws IOException

Get a stream to write the data to dora cache cluster.

Parameters:

alluxioPath - the alluxio path to be written

fsContext - the file system context

outStreamOptions - the output stream options

ufsOutStream - the UfsOutStream for writing data to UFS

uuid - the UUID for a certain FileOutStream

Returns:

the output stream

Throws:

IOException

getChunkSize

protected long getChunkSize()

createNettyPositionReader

public DoraCachePositionReader createNettyPositionReader(URIStatus status,
                                                         Protocol.OpenUfsBlockOptions ufsOptions,
                                                         Optional<CloseableSupplier<PositionReader>> externalPositionReader)

Parameters:

status -

ufsOptions -

externalPositionReader -

Returns:

a netty position reader

createGrpcDataReader

protected GrpcDataReader.Factory createGrpcDataReader(WorkerNetAddress workerNetAddress,
                                                      Protocol.OpenUfsBlockOptions ufsOptions)

createNettyDataReader

protected NettyDataReader createNettyDataReader(WorkerNetAddress workerNetAddress,
                                                Protocol.OpenUfsBlockOptions ufsOptions)

listStatus

public List<URIStatus> listStatus(String path,
                                  ListStatusPOptions options)
                           throws PermissionDeniedException

List Status from Worker.

Parameters:

path -

options -

Returns:

list of URIStatus

Throws:

RuntimeException

PermissionDeniedException

getStatus

public URIStatus getStatus(String path,
                           GetStatusPOptions options)
                    throws PermissionDeniedException

Get status.

Parameters:

path -

options -

Returns:

uri status

Throws:

PermissionDeniedException

getStatusByGrpc

protected URIStatus getStatusByGrpc(String path,
                                    GetStatusPOptions options)
                             throws PermissionDeniedException

Throws:

PermissionDeniedException

createFile

public Pair<URIStatus,String> createFile(String path,
                                         CreateFilePOptions options)
                                  throws PermissionDeniedException

Create File.

Parameters:

path - the file path

options - the option for creating operation

Returns:

URIStatus of new file

Throws:

RuntimeException

PermissionDeniedException

checkFileLocation

public Map<String,List<WorkerNetAddress>> checkFileLocation(String path,
                                                            GetStatusPOptions options)
                                                     throws IOException

Get a map that maps file path to the workers list.

Parameters:

path - the file path to check

options - the get status options

Returns:

a map that maps file path to the workers list

Throws:

IOException

completeFile

public void completeFile(String path,
                         CompleteFilePOptions options,
                         String uuid)
                  throws PermissionDeniedException

Mark the newly created and written file as complete. This is called when out stream is closed. This is equivalent to close() in some file system.

Parameters:

path - The file path

options - the close option

uuid - the uuid of its open file handle

Throws:

PermissionDeniedException

delete

public void delete(String path,
                   DeletePOptions options)
            throws PermissionDeniedException

Delete a file.

Parameters:

path -

options -

Throws:

PermissionDeniedException

rename

public void rename(String src,
                   String dst,
                   RenamePOptions options)
            throws PermissionDeniedException

Rename a src file/dir to dst file/dir.

Parameters:

src - the source file/dir

dst - the destination file/dir

options - the rename option

Throws:

PermissionDeniedException

createDirectory

public void createDirectory(String path,
                            CreateDirectoryPOptions options)
                     throws PermissionDeniedException

Create a dir.

Parameters:

path - the name of the dir

options - the option of this operation

Throws:

PermissionDeniedException

exists

public boolean exists(String path,
                      ExistsPOptions options)
               throws InvalidPathException,
                      IOException,
                      AlluxioException

Check existence of a file or dir.

Parameters:

path - the path of the file or dir

options - the option of this operation

Returns:

true if the file/dir exists, false otherwise

Throws:

InvalidPathException

IOException

AlluxioException

setAttribute

public void setAttribute(String path,
                         SetAttributePOptions options)
                  throws FileDoesNotExistException,
                         IOException,
                         AlluxioException

Set attributes for a file or dir.

Parameters:

path - the path of the file or dir

options - the option of this operation

Throws:

FileDoesNotExistException

IOException

AlluxioException

getWorkerNetAddress

public WorkerNetAddress getWorkerNetAddress(String path)

Get the worker net address of the specific file path.

Parameters:

path - the file path

Returns:

the related worker net address where file locates

choosePreferredWorker

protected BlockWorkerInfo choosePreferredWorker(List<BlockWorkerInfo> workers)

Chooses a client preferred worker from multiple workers which hold multiple replicas.

Parameters:

workers - a list of workers

Returns:

the preferred worker

getContext

public FileSystemContext getContext()

Get Context.

Returns:

a file system context