alluxio.client.file

Class RetryHandlingFileSystemMasterClient

java.lang.Object

alluxio.AbstractClient

alluxio.AbstractMasterClient

alluxio.client.file.RetryHandlingFileSystemMasterClient

All Implemented Interfaces:

Client, FileSystemMasterClient, Closeable, AutoCloseable

@ThreadSafe
public final class RetryHandlingFileSystemMasterClient
extends AbstractMasterClient
implements FileSystemMasterClient

A wrapper for the gRPC client to interact with the file system master, used by alluxio clients.

Nested Class Summary

Nested classes/interfaces inherited from class alluxio.AbstractClient

AbstractClient.RpcCallable<V>

Nested classes/interfaces inherited from interface alluxio.client.file.FileSystemMasterClient

FileSystemMasterClient.Factory

Field Summary

Fields inherited from class alluxio.AbstractClient

mChannel, mClosed, mConnected, mContext, mServerAddress, mServiceVersion, mVersionService

Constructor Summary

Constructors

Constructor and Description
RetryHandlingFileSystemMasterClient(MasterClientContext conf) Creates a new RetryHandlingFileSystemMasterClient instance.

Method Summary

Modifier and Type	Method and Description
protected void	afterConnect() This method is called after the connection is made to the remote.
void	checkAccess(AlluxioURI path, CheckAccessPOptions options) Check access to a path.
List<AlluxioURI>	checkConsistency(AlluxioURI path, CheckConsistencyPOptions options) Checks the consistency of Alluxio metadata against the under storage for all files and directories in a given subtree.
void	completeFile(AlluxioURI path, CompleteFilePOptions options) Marks a file as completed.
void	createDirectory(AlluxioURI path, CreateDirectoryPOptions options) Creates a new directory.
URIStatus	createFile(AlluxioURI path, CreateFilePOptions options) Creates a new file.
void	delete(AlluxioURI path, DeletePOptions options) Deletes a file or a directory.
boolean	exists(AlluxioURI path, ExistsPOptions options) Checks whether a file or directory exists.
void	free(AlluxioURI path, FreePOptions options) Frees a file.
String	getFilePath(long fileId)
String	getJobProgress(JobDescription jobDescription, JobProgressReportFormat format, boolean verbose) Get progress of a job.
Map<String,MountPointInfo>	getMountTable(boolean checkUfs) Lists all mount points and their corresponding under storage addresses.
long	getNewBlockIdForFile(AlluxioURI path)
protected ServiceType	getRemoteServiceType()
protected String	getServiceName()
protected long	getServiceVersion()
List<String>	getStateLockHolders()
URIStatus	getStatus(AlluxioURI path, GetStatusPOptions options)
List<SyncPointInfo>	getSyncPathList() get the list of paths that are currently being actively synced.
void	iterateStatus(AlluxioURI path, ListStatusPOptions options, java.util.function.Consumer<? super URIStatus> action) Performs a specific action on each URIStatus in the result of FileSystemMasterClient.listStatus(alluxio.AlluxioURI, alluxio.grpc.ListStatusPOptions).
List<URIStatus>	listStatus(AlluxioURI path, ListStatusPOptions options)
ListStatusPartialResult	listStatusPartial(AlluxioURI path, ListStatusPartialPOptions options)
void	mount(AlluxioURI alluxioPath, AlluxioURI ufsPath, MountPOptions options) Mounts the given UFS path under the given Alluxio path.
void	needsSync(AlluxioURI path) Mark a path as needed synchronization with the UFS, when this path or any of its children are accessed, a sync with the UFS will be performed.
void	rename(AlluxioURI src, AlluxioURI dst) Renames a file or a directory.
void	rename(AlluxioURI src, AlluxioURI dst, RenamePOptions options) Renames a file or a directory.
AlluxioURI	reverseResolve(AlluxioURI ufsUri) Reverse resolve a ufs uri.
void	scheduleAsyncPersist(AlluxioURI path, ScheduleAsyncPersistencePOptions options) Schedules the async persistence of the given file.
void	setAcl(AlluxioURI path, SetAclAction action, List<AclEntry> entries, SetAclPOptions options) Sets the ACL for a path.
void	setAttribute(AlluxioURI path, SetAttributePOptions options) Sets the file or directory attributes.
void	startSync(AlluxioURI path) Start the active syncing process for a specified path.
boolean	stopJob(JobDescription jobDescription) Stop a job.
void	stopSync(AlluxioURI path) Stop the active syncing process for a specified path.
Optional<String>	submitJob(JobRequest job) Submit a job to scheduler.
void	unmount(AlluxioURI alluxioPath) Unmounts the given Alluxio path.
void	updateMount(AlluxioURI alluxioPath, MountPOptions options) Updates options of a mount point for the given Alluxio path.
void	updateUfsMode(AlluxioURI ufsUri, UpdateUfsModePOptions options) Updates the operation mode for the given ufs path.

Methods inherited from class alluxio.AbstractMasterClient

afterDisconnect, beforeConnect, getConfAddress, queryGrpcServerAddress

Methods inherited from class alluxio.AbstractClient

beforeDisconnect, checkVersion, close, connect, connectWithRuntimeException, disconnect, getRemoteHostName, getRemoteServiceVersion, getRemoteSockAddress, isClosed, isConnected, retryRPC, retryRPC

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface alluxio.client.file.FileSystemMasterClient

getMountTable

Methods inherited from interface alluxio.Client

connect, disconnect, getConfAddress, getRemoteHostName, getRemoteSockAddress, isClosed, isConnected

Methods inherited from interface java.io.Closeable

close

Constructor Detail

RetryHandlingFileSystemMasterClient

public RetryHandlingFileSystemMasterClient(MasterClientContext conf)

Creates a new RetryHandlingFileSystemMasterClient instance.

Parameters:

conf - master client configuration

Method Detail

getRemoteServiceType

protected ServiceType getRemoteServiceType()

Specified by:

getRemoteServiceType in class AbstractClient

Returns:

the expected type of remote service

getServiceName

protected String getServiceName()

Specified by:

getServiceName in class AbstractClient

Returns:

a string representing the specific service

getServiceVersion

protected long getServiceVersion()

Specified by:

getServiceVersion in class AbstractClient

Returns:

the client service version

afterConnect

protected void afterConnect()

Description copied from class: AbstractClient

This method is called after the connection is made to the remote. Implementations should create internal state to finish the connection process.

Overrides:

afterConnect in class AbstractClient

checkAccess

public void checkAccess(AlluxioURI path,
                        CheckAccessPOptions options)
                 throws AlluxioStatusException

Description copied from interface: FileSystemMasterClient

Check access to a path.

Specified by:

checkAccess in interface FileSystemMasterClient

Parameters:

path - the path to check

options - method options

Throws:

AlluxioStatusException

checkConsistency

public List<AlluxioURI> checkConsistency(AlluxioURI path,
                                         CheckConsistencyPOptions options)
                                  throws AlluxioStatusException

Description copied from interface: FileSystemMasterClient

Checks the consistency of Alluxio metadata against the under storage for all files and directories in a given subtree.

Specified by:

checkConsistency in interface FileSystemMasterClient

Parameters:

path - the root of the subtree to check

options - method options

Returns:

a list of inconsistent files and directories

Throws:

AlluxioStatusException

createDirectory

public void createDirectory(AlluxioURI path,
                            CreateDirectoryPOptions options)
                     throws AlluxioStatusException

Description copied from interface: FileSystemMasterClient

Creates a new directory.

Specified by:

createDirectory in interface FileSystemMasterClient

Parameters:

path - the directory path

options - method options

Throws:

AlreadyExistsException - if the directory already exists

AlluxioStatusException

createFile

public URIStatus createFile(AlluxioURI path,
                            CreateFilePOptions options)
                     throws AlluxioStatusException

Description copied from interface: FileSystemMasterClient

Creates a new file.

Specified by:

createFile in interface FileSystemMasterClient

Parameters:

path - the file path

options - method options

Returns:

the uri status of the newly created file

Throws:

AlreadyExistsException - if the file already exists

AlluxioStatusException

completeFile

public void completeFile(AlluxioURI path,
                         CompleteFilePOptions options)
                  throws AlluxioStatusException

Description copied from interface: FileSystemMasterClient

Marks a file as completed.

Specified by:

completeFile in interface FileSystemMasterClient

Parameters:

path - the file path

options - the method options

Throws:

AlluxioStatusException

delete

public void delete(AlluxioURI path,
                   DeletePOptions options)
            throws AlluxioStatusException

Description copied from interface: FileSystemMasterClient

Deletes a file or a directory.

Specified by:

delete in interface FileSystemMasterClient

Parameters:

path - the path to delete

options - method options

Throws:

AlluxioStatusException

exists

public boolean exists(AlluxioURI path,
                      ExistsPOptions options)
               throws AlluxioStatusException

Description copied from interface: FileSystemMasterClient

Checks whether a file or directory exists.

Specified by:

exists in interface FileSystemMasterClient

Parameters:

path - the file path to check existence

options - the method options

Returns:

whether the file path exists

Throws:

AlluxioStatusException

free

public void free(AlluxioURI path,
                 FreePOptions options)
          throws AlluxioStatusException

Description copied from interface: FileSystemMasterClient

Frees a file.

Specified by:

free in interface FileSystemMasterClient

Parameters:

path - the path to free

options - method options

Throws:

NotFoundException - if the path does not exist

AlluxioStatusException

getFilePath

public String getFilePath(long fileId)
                   throws AlluxioStatusException

Specified by:

getFilePath in interface FileSystemMasterClient

Parameters:

fileId - a file id

Returns:

the file path for the given file id

Throws:

AlluxioStatusException

getStatus

public URIStatus getStatus(AlluxioURI path,
                           GetStatusPOptions options)
                    throws AlluxioStatusException

Specified by:

getStatus in interface FileSystemMasterClient

Parameters:

path - the file path

options - the getStatus options

Returns:

the file info for the given file id

Throws:

NotFoundException - if the path does not exist

AlluxioStatusException

getSyncPathList

public List<SyncPointInfo> getSyncPathList()
                                    throws AlluxioStatusException

Description copied from interface: FileSystemMasterClient

get the list of paths that are currently being actively synced.

Specified by:

getSyncPathList in interface FileSystemMasterClient

Returns:

the list of paths

Throws:

AlluxioStatusException

getNewBlockIdForFile

public long getNewBlockIdForFile(AlluxioURI path)
                          throws AlluxioStatusException

Specified by:

getNewBlockIdForFile in interface FileSystemMasterClient

Parameters:

path - the file path

Returns:

the next blockId for the file

Throws:

AlluxioStatusException

getMountTable

public Map<String,MountPointInfo> getMountTable(boolean checkUfs)
                                         throws AlluxioStatusException

Description copied from interface: FileSystemMasterClient

Lists all mount points and their corresponding under storage addresses.

Specified by:

getMountTable in interface FileSystemMasterClient

Parameters:

checkUfs - whether to get UFS usage info

Returns:

a map from String to MountPointInfo

Throws:

AlluxioStatusException

iterateStatus

public void iterateStatus(AlluxioURI path,
                          ListStatusPOptions options,
                          java.util.function.Consumer<? super URIStatus> action)
                   throws AlluxioStatusException

Description copied from interface: FileSystemMasterClient

Performs a specific action on each URIStatus in the result of FileSystemMasterClient.listStatus(alluxio.AlluxioURI, alluxio.grpc.ListStatusPOptions). This method is preferred when iterating over directories with a large number of files or sub-directories inside. The caller can proceed with partial result without waiting for all result returned.

Specified by:

iterateStatus in interface FileSystemMasterClient

Parameters:

path - the path to list information about

options - options to associate with this operation

action - action to apply on each URIStatus

Throws:

NotFoundException - if the path does not exist

AlluxioStatusException

listStatus

public List<URIStatus> listStatus(AlluxioURI path,
                                  ListStatusPOptions options)
                           throws AlluxioStatusException

Specified by:

listStatus in interface FileSystemMasterClient

Parameters:

path - the path to list

options - the listStatus options

Returns:

the list of file information for the given path

Throws:

NotFoundException - if the path does not exist

AlluxioStatusException

listStatusPartial

public ListStatusPartialResult listStatusPartial(AlluxioURI path,
                                                 ListStatusPartialPOptions options)
                                          throws AlluxioStatusException

Specified by:

listStatusPartial in interface FileSystemMasterClient

Parameters:

path - the path to list

options - the listStatus partial options

Returns:

the list of file information for the given path

Throws:

NotFoundException - if the path does not exist

AlluxioStatusException

mount

public void mount(AlluxioURI alluxioPath,
                  AlluxioURI ufsPath,
                  MountPOptions options)
           throws AlluxioStatusException

Description copied from interface: FileSystemMasterClient

Mounts the given UFS path under the given Alluxio path.

Specified by:

mount in interface FileSystemMasterClient

Parameters:

alluxioPath - the Alluxio path

ufsPath - the UFS path

options - mount options

Throws:

AlluxioStatusException

updateMount

public void updateMount(AlluxioURI alluxioPath,
                        MountPOptions options)
                 throws AlluxioStatusException

Description copied from interface: FileSystemMasterClient

Updates options of a mount point for the given Alluxio path.

Specified by:

updateMount in interface FileSystemMasterClient

Parameters:

alluxioPath - the Alluxio path

options - mount options

Throws:

AlluxioStatusException

rename

public void rename(AlluxioURI src,
                   AlluxioURI dst)
            throws AlluxioStatusException

Description copied from interface: FileSystemMasterClient

Renames a file or a directory.

Specified by:

rename in interface FileSystemMasterClient

Parameters:

src - the path to rename

dst - new file path

Throws:

NotFoundException - if the path does not exist

AlluxioStatusException

rename

public void rename(AlluxioURI src,
                   AlluxioURI dst,
                   RenamePOptions options)
            throws AlluxioStatusException

Description copied from interface: FileSystemMasterClient

Renames a file or a directory.

Specified by:

rename in interface FileSystemMasterClient

Parameters:

src - the path to rename

dst - new file path

options - rename options

Throws:

NotFoundException - if the path does not exist

AlluxioStatusException

reverseResolve

public AlluxioURI reverseResolve(AlluxioURI ufsUri)
                          throws AlluxioStatusException

Description copied from interface: FileSystemMasterClient

Reverse resolve a ufs uri.

Specified by:

reverseResolve in interface FileSystemMasterClient

Parameters:

ufsUri - the ufs uri

Returns:

the alluxio path for the ufsUri

Throws:

AlluxioStatusException

setAcl

public void setAcl(AlluxioURI path,
                   SetAclAction action,
                   List<AclEntry> entries,
                   SetAclPOptions options)
            throws AlluxioStatusException

Description copied from interface: FileSystemMasterClient

Sets the ACL for a path.

Specified by:

setAcl in interface FileSystemMasterClient

Parameters:

path - the file or directory path

action - the set action to perform

entries - the ACL entries to use

options - the options for setting ACL

Throws:

NotFoundException - if the path does not exist

AlluxioStatusException

setAttribute

public void setAttribute(AlluxioURI path,
                         SetAttributePOptions options)
                  throws AlluxioStatusException

Description copied from interface: FileSystemMasterClient

Sets the file or directory attributes.

Specified by:

setAttribute in interface FileSystemMasterClient

Parameters:

path - the file or directory path

options - the file or directory attribute options to be set

Throws:

NotFoundException - if the path does not exist

AlluxioStatusException

scheduleAsyncPersist

public void scheduleAsyncPersist(AlluxioURI path,
                                 ScheduleAsyncPersistencePOptions options)
                          throws AlluxioStatusException

Description copied from interface: FileSystemMasterClient

Schedules the async persistence of the given file.

Specified by:

scheduleAsyncPersist in interface FileSystemMasterClient

Parameters:

path - the file path

options - options to use when scheduling the persist

Throws:

AlluxioStatusException

startSync

public void startSync(AlluxioURI path)
               throws AlluxioStatusException

Description copied from interface: FileSystemMasterClient

Start the active syncing process for a specified path.

Specified by:

startSync in interface FileSystemMasterClient

Parameters:

path - the file or directory to be synced

Throws:

AlluxioStatusException

stopSync

public void stopSync(AlluxioURI path)
              throws AlluxioStatusException

Description copied from interface: FileSystemMasterClient

Stop the active syncing process for a specified path.

Specified by:

stopSync in interface FileSystemMasterClient

Parameters:

path - the file or directory to stop syncing

Throws:

AlluxioStatusException

unmount

public void unmount(AlluxioURI alluxioPath)
             throws AlluxioStatusException

Description copied from interface: FileSystemMasterClient

Unmounts the given Alluxio path.

Specified by:

unmount in interface FileSystemMasterClient

Parameters:

alluxioPath - the Alluxio path

Throws:

AlluxioStatusException

updateUfsMode

public void updateUfsMode(AlluxioURI ufsUri,
                          UpdateUfsModePOptions options)
                   throws AlluxioStatusException

Description copied from interface: FileSystemMasterClient

Updates the operation mode for the given ufs path. The path is required to be the scheme and authority only. For example, to update the mode for under storage at hdfs://ns/folder1 specify the argument as hdfs://ns/. Note: the mode for any other mounted under storage which shares the prefix (such as hdfs://ns/folder2) is also updated.

Specified by:

updateUfsMode in interface FileSystemMasterClient

Parameters:

ufsUri - the ufs path

options - the options to update ufs operation mode

Throws:

AlluxioStatusException

getStateLockHolders

public List<String> getStateLockHolders()
                                 throws AlluxioStatusException

Specified by:

getStateLockHolders in interface FileSystemMasterClient

Returns:

the state lock waiters and holders thread identifiers

Throws:

AlluxioStatusException

needsSync

public void needsSync(AlluxioURI path)
               throws AlluxioStatusException

Description copied from interface: FileSystemMasterClient

Mark a path as needed synchronization with the UFS, when this path or any of its children are accessed, a sync with the UFS will be performed.

Specified by:

needsSync in interface FileSystemMasterClient

Parameters:

path - the path to invalidate

Throws:

AlluxioStatusException

submitJob

public Optional<String> submitJob(JobRequest job)

Description copied from interface: FileSystemMasterClient

Submit a job to scheduler.

Specified by:

submitJob in interface FileSystemMasterClient

Parameters:

job - the job request to submit

Returns:

jobId if job is submitted, empty if a job already exists

stopJob

public boolean stopJob(JobDescription jobDescription)

Description copied from interface: FileSystemMasterClient

Stop a job.

Specified by:

stopJob in interface FileSystemMasterClient

Parameters:

jobDescription - job description be stopped

Returns:

true if job is stopped, false if we cannot find job

getJobProgress

public String getJobProgress(JobDescription jobDescription,
                             JobProgressReportFormat format,
                             boolean verbose)

Description copied from interface: FileSystemMasterClient

Get progress of a job.

Specified by:

getJobProgress in interface FileSystemMasterClient

Parameters:

jobDescription - job description to get progress

format - progress report format

verbose - whether to return verbose report

Returns:

the job progress