alluxio.master.file

Class DefaultFileSystemMaster

java.lang.Object

alluxio.master.AbstractMaster

alluxio.master.CoreMaster

alluxio.master.file.DefaultFileSystemMaster

All Implemented Interfaces:

Reconfigurable, FileSystemMaster, Checkpointed, DelegatingJournaled, Journaled, JournalEntryIterable, Master, Server<Boolean>

@NotThreadSafe
public final class DefaultFileSystemMaster
extends CoreMaster
implements FileSystemMaster, DelegatingJournaled, Reconfigurable

The master that handles all file system metadata management.

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static class	DefaultFileSystemMaster.Metrics Class that contains metrics for FileSystemMaster.

Field Summary

Fields inherited from class alluxio.master.CoreMaster

mBackupManager, mJournalSystem, mPort, mSafeModeManager, mStartTimeMs

Fields inherited from class alluxio.master.AbstractMaster

mClock, mMasterContext

Constructor Summary

Constructors

Constructor and Description
DefaultFileSystemMaster(BlockMaster blockMaster, CoreMasterContext masterContext) Creates a new instance of DefaultFileSystemMaster.
DefaultFileSystemMaster(BlockMaster blockMaster, CoreMasterContext masterContext, ExecutorServiceFactory executorServiceFactory) Creates a new instance of DefaultFileSystemMaster.

Method Summary

Modifier and Type	Method and Description
void	activeSyncMetadata(AlluxioURI path, Collection<AlluxioURI> changedFiles, ExecutorService executorService) Actively sync metadata, based on a list of changed files.
void	checkAccess(AlluxioURI path, CheckAccessContext context) Checks access to path.
List<AlluxioURI>	checkConsistency(AlluxioURI path, CheckConsistencyContext context) Checks the consistency of the files and directories in the subtree under the path.
void	cleanupUfs() Periodically clean up the under file systems.
void	close() Closes the server.
void	completeFile(AlluxioURI path, CompleteFileContext context) Completes a file.
long	createDirectory(AlluxioURI path, CreateDirectoryContext context) Creates a directory for a given path.
FileInfo	createFile(AlluxioURI path, CreateFileContext context) Creates a file (not a directory) for a given path.
RpcContext	createRpcContext()
RpcContext	createRpcContext(OperationContext operationContext)
void	delete(AlluxioURI path, DeleteContext context) Deletes a given path.
void	deleteInternal(RpcContext rpcContext, LockedInodePath inodePath, DeleteContext deleteContext) Implements file deletion.
void	free(AlluxioURI path, FreeContext context) Frees or evicts all of the blocks of the file from alluxio storage.
Journaled	getDelegate()
Set<Class<? extends Server>>	getDependencies()
MountPointInfo	getDisplayMountPointInfo(AlluxioURI path) Gets the mount point information of an Alluxio path for display purpose.
List<FileBlockInfo>	getFileBlockInfoList(AlluxioURI path) Gets the FileBlockInfo for all blocks of a file.
long	getFileId(AlluxioURI path) Returns the file id for a given path.
FileInfo	getFileInfo(AlluxioURI path, GetStatusContext context) Returns the FileInfo for a given path.
FileInfo	getFileInfo(long fileId) Returns the FileInfo for a given file id.
FileSystemMasterView	getFileSystemMasterView()
List<AlluxioURI>	getInAlluxioFiles()
List<AlluxioURI>	getInMemoryFiles()
long	getInodeCount() Get the total inode count.
List<Long>	getLostFiles()
long	getMountIdFromUfsPath(AlluxioURI ufsPath) Returns the mount id according to the ufs path.
Map<String,MountPointInfo>	getMountPointInfoSummary()
String	getName()
long	getNewBlockIdForFile(AlluxioURI path) Gets a new block id for the next block of a given file to write to.
AlluxioURI	getPath(long fileId) Gets the path of a file with the given id.
PersistenceState	getPersistenceState(long fileId) Returns the persistence state for a file id.
Set<Long>	getPinIdList()
String	getRootInodeOwner()
Map<ServiceType,GrpcService>	getServices()
List<String>	getStateLockSharedWaitersAndHolders()
List<SyncPointInfo>	getSyncPathList()
List<TimeSeries>	getTimeSeries()
String	getUfsAddress()
UfsInfo	getUfsInfo(long mountId)
List<String>	getWhiteList()
List<WorkerInfo>	getWorkerInfoList()
List<FileInfo>	listStatus(AlluxioURI path, ListStatusContext context) Returns a list of FileInfo for a given path.
void	listStatus(AlluxioURI path, ListStatusContext context, ResultStream<FileInfo> resultStream) Enumerates given path to given batch tracker.
void	mount(AlluxioURI alluxioPath, AlluxioURI ufsPath, MountContext context) Mounts a UFS path onto an Alluxio path.
boolean	recordActiveSyncTxid(long txId, long mountId) Journal the active sync transaction id so that we can restart more efficiently.
void	rename(AlluxioURI srcPath, AlluxioURI dstPath, RenameContext context) Renames a file to a destination.
AlluxioURI	reverseResolve(AlluxioURI ufsUri) Reverse path resolve a ufs uri to an Alluxio path.
void	scheduleAsyncPersistence(AlluxioURI path, ScheduleAsyncPersistenceContext context) Schedules a file for async persistence.
void	setAcl(AlluxioURI path, SetAclAction action, List<AclEntry> entries, SetAclContext context) Sets the ACL for a path.
void	setAttribute(AlluxioURI path, SetAttributeContext context) Sets the file attribute.
protected void	setAttributeSingleFile(RpcContext rpcContext, LockedInodePath inodePath, boolean updateUfs, long opTimeMs, SetAttributeContext context)
void	start(Boolean isPrimary) Starts the Alluxio server.
void	startSync(AlluxioURI syncPoint) starts active sync on a specified alluxioURI.
void	stop() Stops the Alluxio server.
void	stopSync(AlluxioURI syncPoint) stops active sync on a specific syncpoint.
void	unmount(AlluxioURI alluxioPath) Unmounts a UFS path previously mounted onto an Alluxio path.
void	update() When the property changed, this function will be invoked.
void	updateMount(AlluxioURI alluxioPath, MountContext context) Update properties of an Alluxio mount point.
void	updateUfsMode(AlluxioURI ufsPath, UfsMode ufsMode) Update the operation mode for the given ufs path under one or more mount points.
void	validateInodeBlocks(boolean repair) Checks the integrity of the inodes with respect to the blocks of the system.
FileSystemCommand	workerHeartbeat(long workerId, List<Long> persistedFiles, WorkerHeartbeatContext context) Instructs a worker to persist the files.

Methods inherited from class alluxio.master.AbstractMaster

createJournalContext, getExecutorService, getMasterContext

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface alluxio.master.Master

createJournalContext, getMasterContext

Methods inherited from interface alluxio.master.journal.Journaled

applyAndJournal, processJournalEntry, resetState, restoreFromCheckpoint, writeToCheckpoint

Methods inherited from interface alluxio.master.journal.checkpoint.Checkpointed

getCheckpointName

Methods inherited from interface alluxio.master.journal.JournalEntryIterable

getJournalEntryIterator

Methods inherited from interface alluxio.master.journal.DelegatingJournaled

applyAndJournal, getCheckpointName, getJournalEntryIterator, processJournalEntry, resetState, restoreFromCheckpoint, writeToCheckpoint

Constructor Detail

DefaultFileSystemMaster

public DefaultFileSystemMaster(BlockMaster blockMaster,
                               CoreMasterContext masterContext)

Creates a new instance of DefaultFileSystemMaster.

Parameters:

blockMaster - a block master handle

masterContext - the context for Alluxio master

DefaultFileSystemMaster

public DefaultFileSystemMaster(BlockMaster blockMaster,
                               CoreMasterContext masterContext,
                               ExecutorServiceFactory executorServiceFactory)

Creates a new instance of DefaultFileSystemMaster.

Parameters:

blockMaster - a block master handle

masterContext - the context for Alluxio master

executorServiceFactory - a factory for creating the executor service to use for running maintenance threads

Method Detail

getServices

public Map<ServiceType,GrpcService> getServices()

Specified by:

getServices in interface Server<Boolean>

Returns:

a map from service names to gRPC serviced that serve RPCs for this server

getName

public String getName()

Specified by:

getName in interface Server<Boolean>

Returns:

the server's name

getDependencies

public Set<Class<? extends Server>> getDependencies()

Specified by:

getDependencies in interface Server<Boolean>

Overrides:

getDependencies in class AbstractMaster

Returns:

the server dependencies

getDelegate

public Journaled getDelegate()

Specified by:

getDelegate in interface DelegatingJournaled

Returns:

the object to delegate to

start

public void start(Boolean isPrimary)
           throws IOException

Description copied from interface: Server

Starts the Alluxio server.

Specified by:

start in interface Server<Boolean>

Overrides:

start in class AbstractMaster

Parameters:

isPrimary - the start options

Throws:

IOException

stop

public void stop()
          throws IOException

Description copied from interface: Server

Stops the Alluxio server. Here, anything created or started in #start(T) should be cleaned up and shutdown.

Specified by:

stop in interface Server<Boolean>

Overrides:

stop in class AbstractMaster

Throws:

IOException

close

public void close()
           throws IOException

Description copied from interface: Server

Closes the server.

Specified by:

close in interface Server<Boolean>

Overrides:

close in class AbstractMaster

Throws:

IOException

validateInodeBlocks

public void validateInodeBlocks(boolean repair)
                         throws UnavailableException

Description copied from interface: FileSystemMaster

Checks the integrity of the inodes with respect to the blocks of the system.

Specified by:

validateInodeBlocks in interface FileSystemMaster

Parameters:

repair - if true, will attempt to repair the state of the system when inconsistencies are discovered

Throws:

UnavailableException - if the repair attempt fails

cleanupUfs

public void cleanupUfs()

Description copied from interface: FileSystemMaster

Periodically clean up the under file systems.

Specified by:

cleanupUfs in interface FileSystemMaster

getFileId

public long getFileId(AlluxioURI path)
               throws AccessControlException,
                      UnavailableException

Description copied from interface: FileSystemMaster

Returns the file id for a given path. If the given path does not exist in Alluxio, the method attempts to load it from UFS.

This operation requires users to have READ permission of the path.

Specified by:

getFileId in interface FileSystemMaster

Parameters:

path - the path to get the file id for

Returns:

the file id for a given path, or -1 if there is no file at that path

Throws:

AccessControlException - if permission checking fails

UnavailableException

getFileInfo

public FileInfo getFileInfo(long fileId)
                     throws FileDoesNotExistException,
                            AccessControlException,
                            UnavailableException

Description copied from interface: FileSystemMaster

Returns the FileInfo for a given file id. This method is not user-facing but supposed to be called by other internal servers (e.g., block workers, web UI).

Specified by:

getFileInfo in interface FileSystemMaster

Parameters:

fileId - the file id to get the FileInfo for

Returns:

the FileInfo for the given file

Throws:

FileDoesNotExistException - if the file does not exist

AccessControlException - if permission denied

UnavailableException

getFileInfo

public FileInfo getFileInfo(AlluxioURI path,
                            GetStatusContext context)
                     throws FileDoesNotExistException,
                            InvalidPathException,
                            AccessControlException,
                            IOException

Description copied from interface: FileSystemMaster

Returns the FileInfo for a given path.

This operation requires users to have READ permission on the path.

Specified by:

getFileInfo in interface FileSystemMaster

Parameters:

path - the path to get the FileInfo for

context - the method context

Returns:

the FileInfo for the given file id

Throws:

FileDoesNotExistException - if the file does not exist

InvalidPathException - if the file path is not valid

AccessControlException - if permission checking fails

IOException

getMountIdFromUfsPath

public long getMountIdFromUfsPath(AlluxioURI ufsPath)

Description copied from interface: FileSystemMaster

Returns the mount id according to the ufs path.

Specified by:

getMountIdFromUfsPath in interface FileSystemMaster

Parameters:

ufsPath - the ufs path

Returns:

the corresponding mount id

getPersistenceState

public PersistenceState getPersistenceState(long fileId)
                                     throws FileDoesNotExistException

Description copied from interface: FileSystemMaster

Returns the persistence state for a file id.

Specified by:

getPersistenceState in interface FileSystemMaster

Parameters:

fileId - the file id

Returns:

the PersistenceState for the given file id

Throws:

FileDoesNotExistException - if the file does not exist

listStatus

public void listStatus(AlluxioURI path,
                       ListStatusContext context,
                       ResultStream<FileInfo> resultStream)
                throws AccessControlException,
                       FileDoesNotExistException,
                       InvalidPathException,
                       IOException

Description copied from interface: FileSystemMaster

Enumerates given path to given batch tracker. If the given path is a file, the list only contains a single object. If it is a directory, the resulting list contains all direct children of the directory.

This operation requires users to have READ permission on the path, and also EXECUTE permission on the path if it is a directory.

Specified by:

listStatus in interface FileSystemMaster

Parameters:

path - the path to get the FileInfo list for

context - the method context

resultStream - the stream to receive individual results

Throws:

AccessControlException - if permission checking fails

FileDoesNotExistException - if the file does not exist

InvalidPathException - if the path is invalid

IOException

listStatus

public List<FileInfo> listStatus(AlluxioURI path,
                                 ListStatusContext context)
                          throws AccessControlException,
                                 FileDoesNotExistException,
                                 InvalidPathException,
                                 IOException

Description copied from interface: FileSystemMaster

Returns a list of FileInfo for a given path. If the given path is a file, the list only contains a single object. If it is a directory, the resulting list contains all direct children of the directory.

This operation requires users to have READ permission on the path, and also EXECUTE permission on the path if it is a directory.

Specified by:

listStatus in interface FileSystemMaster

Parameters:

path - the path to get the FileInfo list for

context - the method context

Returns:

the list of FileInfos

Throws:

AccessControlException - if permission checking fails

FileDoesNotExistException - if the file does not exist

InvalidPathException - if the path is invalid

IOException

getFileSystemMasterView

public FileSystemMasterView getFileSystemMasterView()

Specified by:

getFileSystemMasterView in interface FileSystemMaster

Returns:

a read-only view of the file system master

checkAccess

public void checkAccess(AlluxioURI path,
                        CheckAccessContext context)
                 throws FileDoesNotExistException,
                        InvalidPathException,
                        AccessControlException,
                        IOException

Description copied from interface: FileSystemMaster

Checks access to path.

Specified by:

checkAccess in interface FileSystemMaster

Parameters:

path - the path to check access to

context - the method context

Throws:

FileDoesNotExistException - if the file does not exist

InvalidPathException - if the given path is invalid

AccessControlException - if permission checking fails

IOException

checkConsistency

public List<AlluxioURI> checkConsistency(AlluxioURI path,
                                         CheckConsistencyContext context)
                                  throws AccessControlException,
                                         FileDoesNotExistException,
                                         InvalidPathException,
                                         IOException

Description copied from interface: FileSystemMaster

Checks the consistency of the files and directories in the subtree under the path.

Specified by:

checkConsistency in interface FileSystemMaster

Parameters:

path - the root of the subtree to check

context - the context to use for the checkConsistency method

Returns:

a list of paths in Alluxio which are not consistent with the under storage

Throws:

AccessControlException - if the permission checking fails

FileDoesNotExistException - if the path does not exist

InvalidPathException - if the path is invalid

IOException

completeFile

public void completeFile(AlluxioURI path,
                         CompleteFileContext context)
                  throws BlockInfoException,
                         FileDoesNotExistException,
                         InvalidPathException,
                         InvalidFileSizeException,
                         FileAlreadyCompletedException,
                         AccessControlException,
                         UnavailableException

Description copied from interface: FileSystemMaster

Completes a file. After a file is completed, it cannot be written to.

This operation requires users to have WRITE permission on the path.

Specified by:

completeFile in interface FileSystemMaster

Parameters:

path - the file path to complete

context - the method context

Throws:

BlockInfoException - if a block information exception is encountered

FileDoesNotExistException - if the file does not exist

InvalidPathException - if an invalid path is encountered

InvalidFileSizeException - if an invalid file size is encountered

FileAlreadyCompletedException - if the file is already completed

AccessControlException - if permission checking fails

UnavailableException

createFile

public FileInfo createFile(AlluxioURI path,
                           CreateFileContext context)
                    throws AccessControlException,
                           InvalidPathException,
                           FileAlreadyExistsException,
                           BlockInfoException,
                           IOException,
                           FileDoesNotExistException

Description copied from interface: FileSystemMaster

Creates a file (not a directory) for a given path.

This operation requires WRITE permission on the parent of this path.

Specified by:

createFile in interface FileSystemMaster

Parameters:

path - the file to create

context - the method context

Returns:

the file info of the created file

Throws:

AccessControlException - if permission checking fails

InvalidPathException - if an invalid path is encountered

FileAlreadyExistsException - if the file already exists

BlockInfoException - if an invalid block information is encountered

FileDoesNotExistException - if the parent of the path does not exist and the recursive option is false

IOException

getNewBlockIdForFile

public long getNewBlockIdForFile(AlluxioURI path)
                          throws FileDoesNotExistException,
                                 InvalidPathException,
                                 AccessControlException,
                                 UnavailableException

Description copied from interface: FileSystemMaster

Gets a new block id for the next block of a given file to write to.

This operation requires users to have WRITE permission on the path as this API is called when creating a new block for a file.

Specified by:

getNewBlockIdForFile in interface FileSystemMaster

Parameters:

path - the path of the file to get the next block id for

Returns:

the next block id for the given file

Throws:

FileDoesNotExistException - if the file does not exist

InvalidPathException - if the given path is not valid

AccessControlException - if permission checking fails

UnavailableException

getMountPointInfoSummary

public Map<String,MountPointInfo> getMountPointInfoSummary()

Specified by:

getMountPointInfoSummary in interface FileSystemMaster

Returns:

a snapshot of the mount table as a mapping of Alluxio path to MountPointInfo

getDisplayMountPointInfo

public MountPointInfo getDisplayMountPointInfo(AlluxioURI path)
                                        throws InvalidPathException

Description copied from interface: FileSystemMaster

Gets the mount point information of an Alluxio path for display purpose.

Specified by:

getDisplayMountPointInfo in interface FileSystemMaster

Parameters:

path - an Alluxio path which must be a mount point

Returns:

the mount point information

Throws:

InvalidPathException

delete

public void delete(AlluxioURI path,
                   DeleteContext context)
            throws IOException,
                   FileDoesNotExistException,
                   DirectoryNotEmptyException,
                   InvalidPathException,
                   AccessControlException

Description copied from interface: FileSystemMaster

Deletes a given path.

This operation requires user to have WRITE permission on the parent of the path.

Specified by:

delete in interface FileSystemMaster

Parameters:

path - the path to delete

context - method context

Throws:

FileDoesNotExistException - if the file does not exist

DirectoryNotEmptyException - if recursive is false and the file is a nonempty directory

InvalidPathException - if the path is invalid

AccessControlException - if permission checking fails

IOException

deleteInternal

public void deleteInternal(RpcContext rpcContext,
                           LockedInodePath inodePath,
                           DeleteContext deleteContext)
                    throws FileDoesNotExistException,
                           IOException,
                           DirectoryNotEmptyException,
                           InvalidPathException

Implements file deletion.

This method does not delete blocks. Instead, it returns deleted inodes so that their blocks can be deleted after the inode deletion journal entry has been written. We cannot delete blocks earlier because the inode deletion may fail, leaving us with inode containing deleted blocks.

Parameters:

rpcContext - the rpc context

inodePath - the file LockedInodePath

deleteContext - the method optitions

Throws:

FileDoesNotExistException

IOException

DirectoryNotEmptyException

InvalidPathException

getFileBlockInfoList

public List<FileBlockInfo> getFileBlockInfoList(AlluxioURI path)
                                         throws FileDoesNotExistException,
                                                InvalidPathException,
                                                AccessControlException,
                                                UnavailableException

Description copied from interface: FileSystemMaster

Gets the FileBlockInfo for all blocks of a file. If path is a directory, an exception is thrown.

This operation requires the client user to have READ permission on the the path.

Specified by:

getFileBlockInfoList in interface FileSystemMaster

Parameters:

path - the path to get the info for

Returns:

a list of FileBlockInfo for all the blocks of the given path

Throws:

FileDoesNotExistException - if the file does not exist or path is a directory

InvalidPathException - if the path of the given file is invalid

AccessControlException - if permission checking fails

UnavailableException

getInAlluxioFiles

public List<AlluxioURI> getInAlluxioFiles()
                                   throws UnavailableException

Specified by:

getInAlluxioFiles in interface FileSystemMaster

Returns:

absolute paths of all in-Alluxio files

Throws:

UnavailableException

getInMemoryFiles

public List<AlluxioURI> getInMemoryFiles()
                                  throws UnavailableException

Specified by:

getInMemoryFiles in interface FileSystemMaster

Returns:

absolute paths of all in-memory files

Throws:

UnavailableException

createDirectory

public long createDirectory(AlluxioURI path,
                            CreateDirectoryContext context)
                     throws InvalidPathException,
                            FileAlreadyExistsException,
                            IOException,
                            AccessControlException,
                            FileDoesNotExistException

Description copied from interface: FileSystemMaster

Creates a directory for a given path.

This operation requires the client user to have WRITE permission on the parent of the path.

Specified by:

createDirectory in interface FileSystemMaster

Parameters:

path - the path of the directory

context - method context

Returns:

the id of the created directory

Throws:

InvalidPathException - when the path is invalid

FileAlreadyExistsException - when there is already a file at path

AccessControlException - if permission checking fails

FileDoesNotExistException - if the parent of the path does not exist and the recursive option is false

IOException

rename

public void rename(AlluxioURI srcPath,
                   AlluxioURI dstPath,
                   RenameContext context)
            throws FileAlreadyExistsException,
                   FileDoesNotExistException,
                   InvalidPathException,
                   IOException,
                   AccessControlException

Description copied from interface: FileSystemMaster

Renames a file to a destination.

This operation requires users to have WRITE permission on the parent of the src path, and WRITE permission on the parent of the dst path.

Specified by:

rename in interface FileSystemMaster

Parameters:

srcPath - the source path to rename

dstPath - the destination path to rename the file to

context - method context

Throws:

FileAlreadyExistsException - if the file already exists

FileDoesNotExistException - if a non-existent file is encountered

InvalidPathException - if an invalid path is encountered

AccessControlException - if permission checking fails

IOException

free

public void free(AlluxioURI path,
                 FreeContext context)
          throws FileDoesNotExistException,
                 InvalidPathException,
                 AccessControlException,
                 UnexpectedAlluxioException,
                 IOException

Description copied from interface: FileSystemMaster

Frees or evicts all of the blocks of the file from alluxio storage. If the given file is a directory, and the 'recursive' flag is enabled, all descendant files will also be freed.

This operation requires users to have READ permission on the path.

Specified by:

free in interface FileSystemMaster

Parameters:

path - the path to free method

context - context to free method

Throws:

FileDoesNotExistException - if the file does not exist

InvalidPathException - if the given path is invalid

AccessControlException - if permission checking fails

UnexpectedAlluxioException - if the file or directory can not be freed

IOException

getPath

public AlluxioURI getPath(long fileId)
                   throws FileDoesNotExistException

Description copied from interface: FileSystemMaster

Gets the path of a file with the given id.

Specified by:

getPath in interface FileSystemMaster

Parameters:

fileId - the id of the file to look up

Returns:

the path of the file

Throws:

FileDoesNotExistException - raise if the file does not exist

getPinIdList

public Set<Long> getPinIdList()

Specified by:

getPinIdList in interface FileSystemMaster

Returns:

the set of inode ids which are pinned

getUfsAddress

public String getUfsAddress()

Specified by:

getUfsAddress in interface FileSystemMaster

Returns:

the ufs address for this master

getUfsInfo

public UfsInfo getUfsInfo(long mountId)

Specified by:

getUfsInfo in interface FileSystemMaster

Parameters:

mountId - the mount id to query

Returns:

the ufs information for the given mount id

getWhiteList

public List<String> getWhiteList()

Specified by:

getWhiteList in interface FileSystemMaster

Returns:

the white list

getLostFiles

public List<Long> getLostFiles()

Specified by:

getLostFiles in interface FileSystemMaster

Returns:

all the files lost on the workers

updateMount

public void updateMount(AlluxioURI alluxioPath,
                        MountContext context)
                 throws FileAlreadyExistsException,
                        FileDoesNotExistException,
                        InvalidPathException,
                        IOException,
                        AccessControlException

Description copied from interface: FileSystemMaster

Update properties of an Alluxio mount point.

This operation requires users to have WRITE permission on the parent of the Alluxio path.

Specified by:

updateMount in interface FileSystemMaster

Parameters:

alluxioPath - the Alluxio path to update, must be a mount point

context - the mount context

Throws:

FileDoesNotExistException - if the given path does not exist

InvalidPathException - if the given path is not a mount point

AccessControlException - if the permission check fails

FileAlreadyExistsException

IOException

mount

public void mount(AlluxioURI alluxioPath,
                  AlluxioURI ufsPath,
                  MountContext context)
           throws FileAlreadyExistsException,
                  FileDoesNotExistException,
                  InvalidPathException,
                  IOException,
                  AccessControlException

Description copied from interface: FileSystemMaster

Mounts a UFS path onto an Alluxio path.

This operation requires users to have WRITE permission on the parent of the Alluxio path.

Specified by:

mount in interface FileSystemMaster

Parameters:

alluxioPath - the Alluxio path to mount to

ufsPath - the UFS path to mount

context - the mount context

Throws:

FileAlreadyExistsException - if the path to be mounted to already exists

FileDoesNotExistException - if the parent of the path to be mounted to does not exist

InvalidPathException - if an invalid path is encountered

AccessControlException - if the permission check fails

IOException

unmount

public void unmount(AlluxioURI alluxioPath)
             throws FileDoesNotExistException,
                    InvalidPathException,
                    IOException,
                    AccessControlException

Description copied from interface: FileSystemMaster

Unmounts a UFS path previously mounted onto an Alluxio path.

This operation requires users to have WRITE permission on the parent of the Alluxio path.

Specified by:

unmount in interface FileSystemMaster

Parameters:

alluxioPath - the Alluxio path to unmount, must be a mount point

Throws:

FileDoesNotExistException - if the path to be mounted does not exist

InvalidPathException - if the given path is not a mount point

AccessControlException - if the permission check fails

IOException

setAcl

public void setAcl(AlluxioURI path,
                   SetAclAction action,
                   List<AclEntry> entries,
                   SetAclContext context)
            throws FileDoesNotExistException,
                   AccessControlException,
                   InvalidPathException,
                   IOException

Description copied from interface: FileSystemMaster

Sets the ACL for a path.

Specified by:

setAcl in interface FileSystemMaster

Parameters:

path - the path to set attribute for

action - the set action to perform

entries - the list of ACL entries for setting ACL

context - the context for setting ACL

Throws:

FileDoesNotExistException - if the file does not exist

AccessControlException - if permission checking fails

InvalidPathException - if the given path is invalid

IOException

setAttribute

public void setAttribute(AlluxioURI path,
                         SetAttributeContext context)
                  throws FileDoesNotExistException,
                         AccessControlException,
                         InvalidPathException,
                         IOException

Description copied from interface: FileSystemMaster

Sets the file attribute.

This operation requires users to have WRITE permission on the path. In addition, the client user must be a super user when setting the owner, and must be a super user or the owner when setting the group or permission.

Specified by:

setAttribute in interface FileSystemMaster

Parameters:

path - the path to set attribute for

context - master operation context

Throws:

FileDoesNotExistException - if the file does not exist

AccessControlException - if permission checking fails

InvalidPathException - if the given path is invalid

IOException

scheduleAsyncPersistence

public void scheduleAsyncPersistence(AlluxioURI path,
                                     ScheduleAsyncPersistenceContext context)
                              throws AlluxioException,
                                     UnavailableException

Description copied from interface: FileSystemMaster

Schedules a file for async persistence.

Specified by:

scheduleAsyncPersistence in interface FileSystemMaster

Parameters:

path - the path of the file for persistence

context - the schedule async persistence context

Throws:

AlluxioException

UnavailableException

activeSyncMetadata

public void activeSyncMetadata(AlluxioURI path,
                               Collection<AlluxioURI> changedFiles,
                               ExecutorService executorService)
                        throws IOException

Actively sync metadata, based on a list of changed files.

Specified by:

activeSyncMetadata in interface FileSystemMaster

Parameters:

path - the path to sync

changedFiles - collection of files that are changed under the path to sync, if this is null, force sync the entire directory

executorService - executor to execute the parallel incremental sync

Throws:

IOException

recordActiveSyncTxid

public boolean recordActiveSyncTxid(long txId,
                                    long mountId)

Description copied from interface: FileSystemMaster

Journal the active sync transaction id so that we can restart more efficiently.

Specified by:

recordActiveSyncTxid in interface FileSystemMaster

Parameters:

txId - transaction id

mountId - mount id

Returns:

true if successfully recorded in the journal

update

public void update()

Description copied from interface: Reconfigurable

When the property changed, this function will be invoked.

Specified by:

update in interface Reconfigurable

workerHeartbeat

public FileSystemCommand workerHeartbeat(long workerId,
                                         List<Long> persistedFiles,
                                         WorkerHeartbeatContext context)
                                  throws IOException

Description copied from interface: FileSystemMaster

Instructs a worker to persist the files.

Needs WRITE permission on the list of files.

Specified by:

workerHeartbeat in interface FileSystemMaster

Parameters:

workerId - the id of the worker that heartbeats

persistedFiles - the files that persisted on the worker

context - the method context

Returns:

the command for persisting the blocks of a file

Throws:

IOException

setAttributeSingleFile

protected void setAttributeSingleFile(RpcContext rpcContext,
                                      LockedInodePath inodePath,
                                      boolean updateUfs,
                                      long opTimeMs,
                                      SetAttributeContext context)
                               throws FileDoesNotExistException,
                                      InvalidPathException,
                                      AccessControlException

Parameters:

inodePath - the LockedInodePath to use

updateUfs - whether to update the UFS with the attribute change

opTimeMs - the operation time (in milliseconds)

context - the method context

Throws:

FileDoesNotExistException

InvalidPathException

AccessControlException

getSyncPathList

public List<SyncPointInfo> getSyncPathList()

Specified by:

getSyncPathList in interface FileSystemMaster

Returns:

the list of sync path

startSync

public void startSync(AlluxioURI syncPoint)
               throws IOException,
                      InvalidPathException,
                      AccessControlException,
                      ConnectionFailedException

Description copied from interface: FileSystemMaster

starts active sync on a specified alluxioURI.

Specified by:

startSync in interface FileSystemMaster

Parameters:

syncPoint - sync point which is a valid path in Alluxio namespace

Throws:

UnavailableException

InvalidPathException

AccessControlException

IOException

ConnectionFailedException

stopSync

public void stopSync(AlluxioURI syncPoint)
              throws IOException,
                     InvalidPathException,
                     AccessControlException

Description copied from interface: FileSystemMaster

stops active sync on a specific syncpoint.

Specified by:

stopSync in interface FileSystemMaster

Parameters:

syncPoint - alluxio path that has been added as a sync point

Throws:

UnavailableException

InvalidPathException

AccessControlException

IOException

getWorkerInfoList

public List<WorkerInfo> getWorkerInfoList()
                                   throws UnavailableException

Specified by:

getWorkerInfoList in interface FileSystemMaster

Returns:

a list of WorkerInfo objects representing the workers in Alluxio

Throws:

UnavailableException

getInodeCount

public long getInodeCount()

Description copied from interface: FileSystemMaster

Get the total inode count.

Specified by:

getInodeCount in interface FileSystemMaster

Returns:

inode count

updateUfsMode

public void updateUfsMode(AlluxioURI ufsPath,
                          UfsMode ufsMode)
                   throws InvalidPathException,
                          InvalidArgumentException,
                          UnavailableException,
                          AccessControlException

Description copied from interface: FileSystemMaster

Update the operation mode for the given ufs path under one or more mount points.

Specified by:

updateUfsMode in interface FileSystemMaster

Parameters:

ufsPath - the physical ufs path

ufsMode - the ufs operation mode

Throws:

InvalidPathException - if ufs path is not used by any mount point

InvalidArgumentException - if arguments for the method are invalid

UnavailableException

AccessControlException

createRpcContext

public RpcContext createRpcContext()
                            throws UnavailableException

Returns:

a context for executing an RPC

Throws:

UnavailableException

createRpcContext

public RpcContext createRpcContext(OperationContext operationContext)
                            throws UnavailableException

Parameters:

operationContext - the operation context

Returns:

a context for executing an RPC

Throws:

UnavailableException

getTimeSeries

public List<TimeSeries> getTimeSeries()

Specified by:

getTimeSeries in interface FileSystemMaster

Returns:

the time series data stored by the master

reverseResolve

public AlluxioURI reverseResolve(AlluxioURI ufsUri)
                          throws InvalidPathException

Description copied from interface: FileSystemMaster

Reverse path resolve a ufs uri to an Alluxio path.

Specified by:

reverseResolve in interface FileSystemMaster

Parameters:

ufsUri - ufs uri

Returns:

alluxio path

Throws:

InvalidPathException

getRootInodeOwner

@Nullable
public String getRootInodeOwner()

Specified by:

getRootInodeOwner in interface FileSystemMaster

Returns:

the owner of the root inode, null if the inode tree is not initialized

getStateLockSharedWaitersAndHolders

public List<String> getStateLockSharedWaitersAndHolders()

Specified by:

getStateLockSharedWaitersAndHolders in interface FileSystemMaster

Returns:

the list of thread identifiers that are waiting and holding the state lock