alluxio.master.file

Class DefaultFileSystemMaster

java.lang.Object

alluxio.master.AbstractMaster

alluxio.master.CoreMaster

alluxio.master.file.DefaultFileSystemMaster

All Implemented Interfaces:

FileSystemMaster, Checkpointed, DelegatingJournaled, Journaled, JournalEntryIterable, Master, Server<Boolean>

@NotThreadSafe
public class DefaultFileSystemMaster
extends CoreMaster
implements FileSystemMaster, DelegatingJournaled

The master that handles all file system metadata management.

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static class	DefaultFileSystemMaster.Metrics Class that contains metrics for FileSystemMaster.

Field Summary

Fields

Modifier and Type	Field and Description
protected AsyncUserAccessAuditLogWriter	mAsyncAuditLogWriter Log writer for user access audit log.
protected InodeTree	mInodeTree This manages the file system inode structure.
int	mRecursiveOperationForceFlushEntries
protected CallTracker	mStateLockCallTracker Used to check pending/running backup from RPCs.

Fields inherited from class alluxio.master.CoreMaster

mBackupManager, mJournalSystem, mPort, mPrimarySelector, mSafeModeManager, mStartTimeMs

Fields inherited from class alluxio.master.AbstractMaster

mJournal, mMasterContext

Constructor Summary

Constructors

Constructor and Description
DefaultFileSystemMaster(BlockMaster blockMaster, CoreMasterContext masterContext) Creates a new instance of DefaultFileSystemMaster.
DefaultFileSystemMaster(BlockMaster blockMaster, CoreMasterContext masterContext, ExecutorServiceFactory executorServiceFactory, java.time.Clock clock) Creates a new instance of DefaultFileSystemMaster.

Method Summary

Modifier and Type	Method and Description
protected boolean	areDescendantsLoaded(InodeDirectoryView inode)
void	checkAccess(AlluxioURI path, CheckAccessContext context) Checks access to path.
List<AlluxioURI>	checkConsistency(AlluxioURI path, CheckConsistencyContext context) Checks the consistency of the files and directories in the subtree under the path.
void	cleanupUfs() Periodically clean up the under file systems.
void	close() Closes the server.
void	completeFile(AlluxioURI path, CompleteFileContext context) Completes a file.
protected FileSystemMasterAuditContext	createAuditContext(String command, AlluxioURI srcPath, AlluxioURI dstPath, Inode srcInode) Creates a FileSystemMasterAuditContext instance.
protected BlockDeletionContext	createBlockDeletionContext()
long	createDirectory(AlluxioURI path, CreateDirectoryContext context) Creates a directory for a given path.
FileInfo	createFile(AlluxioURI path, CreateFileContext context) Creates a file (not a directory) for a given path.
JournalContext	createJournalContext()
RpcContext	createRpcContext()
RpcContext	createRpcContext(OperationContext operationContext)
protected LockingScheme	createSyncLockingScheme(AlluxioURI path, FileSystemMasterCommonPOptions options, DescendantType descendantType)
void	delete(AlluxioURI path, DeleteContext context) Deletes a given path.
void	deleteInternal(RpcContext rpcContext, LockedInodePath inodePath, DeleteContext deleteContext, boolean bypassPermCheck) Implements file deletion.
protected void	ensureFullPathAndUpdateCache(LockedInodePath inodePath) Checks to see if the entire path exists in Alluxio.
boolean	exists(AlluxioURI path, ExistsContext context) Checks path exists.
void	free(AlluxioURI path, FreeContext context) Frees or evicts all of the blocks of the file from alluxio storage.
Journaled	getDelegate()
Set<Class<? extends Server>>	getDependencies()
MountPointInfo	getDisplayMountPointInfo(AlluxioURI path) Gets the mount point information of an Alluxio path for display purpose.
List<FileBlockInfo>	getFileBlockInfoList(AlluxioURI path) Gets the FileBlockInfo for all blocks of a file.
long	getFileId(AlluxioURI path) Returns the file id for a given path.
FileInfo	getFileInfo(AlluxioURI path, GetStatusContext context) Returns the FileInfo for a given path.
FileInfo	getFileInfo(long fileId) Returns the FileInfo for a given file id.
FileSystemMasterView	getFileSystemMasterView()
List<AlluxioURI>	getInAlluxioFiles()
List<AlluxioURI>	getInMemoryFiles()
long	getInodeCount() Get the total inode count.
Set<Long>	getLostFiles()
List<WorkerInfo>	getLostWorkerList()
long	getMountIdFromUfsPath(AlluxioURI ufsPath) Returns the mount id according to the ufs path.
Map<String,MountPointInfo>	getMountPointInfoSummary(boolean checkUfs)
MountTable	getMountTable()
String	getName()
long	getNewBlockIdForFile(AlluxioURI path) Gets a new block id for the next block of a given file to write to.
AlluxioURI	getPath(long fileId) Gets the path of a file with the given id.
PersistenceState	getPersistenceState(long fileId) Returns the persistence state for a file id.
Set<Long>	getPinIdList()
String	getRootInodeOwner()
Scheduler	getScheduler() Get scheduler.
Map<ServiceType,GrpcService>	getServices()
Collection<String>	getStateLockSharedWaitersAndHolders()
UfsSyncPathCache	getSyncPathCache()
List<TimeSeries>	getTimeSeries()
String	getUfsAddress()
UfsInfo	getUfsInfo(long mountId)
MasterUfsManager	getUfsManager()
List<String>	getWhiteList()
List<WorkerInfo>	getWorkerInfoList()
List<FileInfo>	listStatus(AlluxioURI path, ListStatusContext context) Returns a list of FileInfo for a given path.
void	listStatus(AlluxioURI path, ListStatusContext context, ResultStream<FileInfo> resultStream) Enumerates given path to given batch tracker.
protected void	loadMetadataIfNotExist(RpcContext rpcContext, AlluxioURI path, LoadMetadataContext context) Loads metadata for the path if it is (non-existing || load direct children is set).
void	mount(AlluxioURI alluxioPath, AlluxioURI ufsPath, MountContext context) Mounts a UFS path onto an Alluxio path.
void	needsSync(AlluxioURI path) Mark a path as needed synchronization with the UFS, when this path or any of its children are accessed, a sync with the UFS will be performed.
void	rename(AlluxioURI srcPath, AlluxioURI dstPath, RenameContext context) Renames a file to a destination.
AlluxioURI	reverseResolve(AlluxioURI ufsUri) Reverse path resolve a ufs uri to an Alluxio path.
void	scheduleAsyncPersistence(AlluxioURI path, ScheduleAsyncPersistenceContext context) Schedules a file for async persistence.
void	setAcl(AlluxioURI path, SetAclAction action, List<AclEntry> entries, SetAclContext context) Sets the ACL for a path.
void	setAttribute(AlluxioURI path, SetAttributeContext context) Sets the file attribute.
protected void	setAttributeSingleFile(RpcContext rpcContext, LockedInodePath inodePath, boolean updateUfs, long opTimeMs, SetAttributeContext context)
void	start(Boolean isPrimary) Starts the Alluxio server.
void	stop() Stops the Alluxio server.
void	unmount(AlluxioURI alluxioPath) Unmounts a UFS path previously mounted onto an Alluxio path.
void	updateMount(AlluxioURI alluxioPath, MountContext context) Update properties of an Alluxio mount point.
void	updateUfsMode(AlluxioURI ufsPath, UfsMode ufsMode) Update the operation mode for the given ufs path under one or more mount points.
FileSystemCommand	workerHeartbeat(long workerId, List<Long> persistedFiles, WorkerHeartbeatContext context) Instructs a worker to persist the files.

Methods inherited from class alluxio.master.AbstractMaster

getExecutorService, getMasterContext

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface alluxio.master.file.FileSystemMaster

getMountPointInfoSummary

Methods inherited from interface alluxio.master.Master

getMasterContext, getStandbyServices

Methods inherited from interface alluxio.master.journal.Journaled

applyAndJournal, processJournalEntry, resetState, restoreFromCheckpoint, writeToCheckpoint

Methods inherited from interface alluxio.master.journal.checkpoint.Checkpointed

getCheckpointName, restoreFromCheckpoint, writeToCheckpoint

Methods inherited from interface alluxio.master.journal.JournalEntryIterable

getJournalEntryIterator

Methods inherited from interface alluxio.master.journal.DelegatingJournaled

applyAndJournal, getCheckpointName, getJournalEntryIterator, processJournalEntry, resetState, restoreFromCheckpoint, restoreFromCheckpoint, writeToCheckpoint, writeToCheckpoint

Field Detail

mInodeTree

protected final InodeTree mInodeTree

This manages the file system inode structure. This must be journaled.

mAsyncAuditLogWriter

protected AsyncUserAccessAuditLogWriter mAsyncAuditLogWriter

Log writer for user access audit log.

mStateLockCallTracker

protected final CallTracker mStateLockCallTracker

Used to check pending/running backup from RPCs.

mRecursiveOperationForceFlushEntries

public final int mRecursiveOperationForceFlushEntries

Constructor Detail

DefaultFileSystemMaster

public DefaultFileSystemMaster(BlockMaster blockMaster,
                               CoreMasterContext masterContext)

Creates a new instance of DefaultFileSystemMaster.

Parameters:

blockMaster - a block master handle

masterContext - the context for Alluxio master

DefaultFileSystemMaster

public DefaultFileSystemMaster(BlockMaster blockMaster,
                               CoreMasterContext masterContext,
                               ExecutorServiceFactory executorServiceFactory,
                               java.time.Clock clock)

Creates a new instance of DefaultFileSystemMaster.

Parameters:

blockMaster - a block master handle

masterContext - the context for Alluxio master

executorServiceFactory - a factory for creating the executor service to use for running maintenance threads

clock - the clock used to compute the current time

Method Detail

getServices

public Map<ServiceType,GrpcService> getServices()

Specified by:

getServices in interface Server<Boolean>

Returns:

a map from service names to gRPC serviced that serve RPCs for this server

getName

public String getName()

Specified by:

getName in interface Server<Boolean>

Returns:

the server's name

getDependencies

public Set<Class<? extends Server>> getDependencies()

Specified by:

getDependencies in interface Server<Boolean>

Overrides:

getDependencies in class AbstractMaster

Returns:

the server dependencies

getDelegate

public Journaled getDelegate()

Specified by:

getDelegate in interface DelegatingJournaled

Returns:

the object to delegate to

createJournalContext

public JournalContext createJournalContext()
                                    throws UnavailableException

Specified by:

createJournalContext in interface Master

Overrides:

createJournalContext in class AbstractMaster

Returns:

a journal context for writing journal entries to the master

Throws:

UnavailableException

start

public void start(Boolean isPrimary)
           throws IOException

Description copied from interface: Server

Starts the Alluxio server.

Specified by:

start in interface Server<Boolean>

Overrides:

start in class AbstractMaster

Parameters:

isPrimary - the start options

Throws:

IOException

stop

public void stop()
          throws IOException

Description copied from interface: Server

Stops the Alluxio server. Here, anything created or started in #start(T) should be cleaned up and shutdown.

Specified by:

stop in interface Server<Boolean>

Overrides:

stop in class AbstractMaster

Throws:

IOException

close

public void close()
           throws IOException

Description copied from interface: Server

Closes the server.

Specified by:

close in interface Server<Boolean>

Overrides:

close in class AbstractMaster

Throws:

IOException

cleanupUfs

public void cleanupUfs()

Description copied from interface: FileSystemMaster

Periodically clean up the under file systems.

Specified by:

cleanupUfs in interface FileSystemMaster

getFileId

public long getFileId(AlluxioURI path)
               throws AccessControlException,
                      UnavailableException

Description copied from interface: FileSystemMaster

Returns the file id for a given path. If the given path does not exist in Alluxio, the method attempts to load it from UFS.

This operation requires users to have READ permission of the path.

Specified by:

getFileId in interface FileSystemMaster

Parameters:

path - the path to get the file id for

Returns:

the file id for a given path, or -1 if there is no file at that path

Throws:

AccessControlException - if permission checking fails

UnavailableException

getFileInfo

public FileInfo getFileInfo(long fileId)
                     throws FileDoesNotExistException,
                            AccessControlException,
                            UnavailableException

Description copied from interface: FileSystemMaster

Returns the FileInfo for a given file id. This method is not user-facing but supposed to be called by other internal servers (e.g., block workers, web UI).

Specified by:

getFileInfo in interface FileSystemMaster

Parameters:

fileId - the file id to get the FileInfo for

Returns:

the FileInfo for the given file

Throws:

FileDoesNotExistException - if the file does not exist

AccessControlException - if permission denied

UnavailableException

getFileInfo

public FileInfo getFileInfo(AlluxioURI path,
                            GetStatusContext context)
                     throws FileDoesNotExistException,
                            InvalidPathException,
                            AccessControlException,
                            IOException

Description copied from interface: FileSystemMaster

Returns the FileInfo for a given path.

This operation requires users to have READ permission on the path.

Specified by:

getFileInfo in interface FileSystemMaster

Parameters:

path - the path to get the FileInfo for

context - the method context

Returns:

the FileInfo for the given file id

Throws:

FileDoesNotExistException - if the file does not exist

InvalidPathException - if the file path is not valid

AccessControlException - if permission checking fails

IOException

getMountIdFromUfsPath

public long getMountIdFromUfsPath(AlluxioURI ufsPath)

Description copied from interface: FileSystemMaster

Returns the mount id according to the ufs path.

Specified by:

getMountIdFromUfsPath in interface FileSystemMaster

Parameters:

ufsPath - the ufs path

Returns:

the corresponding mount id

getPersistenceState

public PersistenceState getPersistenceState(long fileId)
                                     throws FileDoesNotExistException

Description copied from interface: FileSystemMaster

Returns the persistence state for a file id.

Specified by:

getPersistenceState in interface FileSystemMaster

Parameters:

fileId - the file id

Returns:

the PersistenceState for the given file id

Throws:

FileDoesNotExistException - if the file does not exist

listStatus

public void listStatus(AlluxioURI path,
                       ListStatusContext context,
                       ResultStream<FileInfo> resultStream)
                throws AccessControlException,
                       FileDoesNotExistException,
                       InvalidPathException,
                       IOException

Description copied from interface: FileSystemMaster

Enumerates given path to given batch tracker. If the given path is a file, the list only contains a single object. If it is a directory, the resulting list contains all direct children of the directory.

This operation requires users to have READ permission on the path, and also EXECUTE permission on the path if it is a directory.

Specified by:

listStatus in interface FileSystemMaster

Parameters:

path - the path to get the FileInfo list for

context - the method context

resultStream - the stream to receive individual results

Throws:

AccessControlException - if permission checking fails

FileDoesNotExistException - if the file does not exist

InvalidPathException - if the path is invalid

IOException

listStatus

public List<FileInfo> listStatus(AlluxioURI path,
                                 ListStatusContext context)
                          throws AccessControlException,
                                 FileDoesNotExistException,
                                 InvalidPathException,
                                 IOException

Description copied from interface: FileSystemMaster

Returns a list of FileInfo for a given path. If the given path is a file, the list only contains a single object. If it is a directory, the resulting list contains all direct children of the directory.

This operation requires users to have READ permission on the path, and also EXECUTE permission on the path if it is a directory.

Specified by:

listStatus in interface FileSystemMaster

Parameters:

path - the path to get the FileInfo list for

context - the method context

Returns:

the list of FileInfos

Throws:

AccessControlException - if permission checking fails

FileDoesNotExistException - if the file does not exist

InvalidPathException - if the path is invalid

IOException

areDescendantsLoaded

protected boolean areDescendantsLoaded(InodeDirectoryView inode)

ensureFullPathAndUpdateCache

protected void ensureFullPathAndUpdateCache(LockedInodePath inodePath)
                                     throws InvalidPathException,
                                            FileDoesNotExistException

Checks to see if the entire path exists in Alluxio. Updates the absent cache if it does not exist.

Parameters:

inodePath - the path to ensure

Throws:

InvalidPathException

FileDoesNotExistException

getFileSystemMasterView

public FileSystemMasterView getFileSystemMasterView()

Specified by:

getFileSystemMasterView in interface FileSystemMaster

Returns:

a read-only view of the file system master

checkAccess

public void checkAccess(AlluxioURI path,
                        CheckAccessContext context)
                 throws FileDoesNotExistException,
                        InvalidPathException,
                        AccessControlException,
                        IOException

Description copied from interface: FileSystemMaster

Checks access to path.

Specified by:

checkAccess in interface FileSystemMaster

Parameters:

path - the path to check access to

context - the method context

Throws:

FileDoesNotExistException - if the file does not exist

InvalidPathException - if the given path is invalid

AccessControlException - if permission checking fails

IOException

checkConsistency

public List<AlluxioURI> checkConsistency(AlluxioURI path,
                                         CheckConsistencyContext context)
                                  throws AccessControlException,
                                         FileDoesNotExistException,
                                         InvalidPathException,
                                         IOException

Description copied from interface: FileSystemMaster

Checks the consistency of the files and directories in the subtree under the path.

Specified by:

checkConsistency in interface FileSystemMaster

Parameters:

path - the root of the subtree to check

context - the context to use for the checkConsistency method

Returns:

a list of paths in Alluxio which are not consistent with the under storage

Throws:

AccessControlException - if the permission checking fails

FileDoesNotExistException - if the path does not exist

InvalidPathException - if the path is invalid

IOException

exists

public boolean exists(AlluxioURI path,
                      ExistsContext context)
               throws AccessControlException,
                      IOException

Description copied from interface: FileSystemMaster

Checks path exists.

Specified by:

exists in interface FileSystemMaster

Parameters:

path - path to check

context - the method context

Returns:

whether the path exists

Throws:

AccessControlException

IOException

completeFile

public void completeFile(AlluxioURI path,
                         CompleteFileContext context)
                  throws BlockInfoException,
                         FileDoesNotExistException,
                         InvalidPathException,
                         InvalidFileSizeException,
                         FileAlreadyCompletedException,
                         AccessControlException,
                         UnavailableException

Description copied from interface: FileSystemMaster

Completes a file. After a file is completed, it cannot be written to.

This operation requires users to have WRITE permission on the path.

Specified by:

completeFile in interface FileSystemMaster

Parameters:

path - the file path to complete

context - the method context

Throws:

BlockInfoException - if a block information exception is encountered

FileDoesNotExistException - if the file does not exist

InvalidPathException - if an invalid path is encountered

InvalidFileSizeException - if an invalid file size is encountered

FileAlreadyCompletedException - if the file is already completed

AccessControlException - if permission checking fails

UnavailableException

createFile

public FileInfo createFile(AlluxioURI path,
                           CreateFileContext context)
                    throws AccessControlException,
                           InvalidPathException,
                           FileAlreadyExistsException,
                           BlockInfoException,
                           IOException,
                           FileDoesNotExistException

Description copied from interface: FileSystemMaster

Creates a file (not a directory) for a given path.

This operation requires WRITE permission on the parent of this path.

Specified by:

createFile in interface FileSystemMaster

Parameters:

path - the file to create

context - the method context

Returns:

the file info of the created file

Throws:

AccessControlException - if permission checking fails

InvalidPathException - if an invalid path is encountered

FileAlreadyExistsException - if the file already exists

BlockInfoException - if an invalid block information is encountered

FileDoesNotExistException - if the parent of the path does not exist and the recursive option is false

IOException

getNewBlockIdForFile

public long getNewBlockIdForFile(AlluxioURI path)
                          throws FileDoesNotExistException,
                                 InvalidPathException,
                                 AccessControlException,
                                 UnavailableException

Description copied from interface: FileSystemMaster

Gets a new block id for the next block of a given file to write to.

This operation requires users to have WRITE permission on the path as this API is called when creating a new block for a file.

Specified by:

getNewBlockIdForFile in interface FileSystemMaster

Parameters:

path - the path of the file to get the next block id for

Returns:

the next block id for the given file

Throws:

FileDoesNotExistException - if the file does not exist

InvalidPathException - if the given path is not valid

AccessControlException - if permission checking fails

UnavailableException

getMountPointInfoSummary

public Map<String,MountPointInfo> getMountPointInfoSummary(boolean checkUfs)

Specified by:

getMountPointInfoSummary in interface FileSystemMaster

Parameters:

checkUfs - if true, invoke ufs to set ufs properties

Returns:

a snapshot of the mount table as a mapping of Alluxio path to MountPointInfo

getDisplayMountPointInfo

public MountPointInfo getDisplayMountPointInfo(AlluxioURI path)
                                        throws InvalidPathException

Description copied from interface: FileSystemMaster

Gets the mount point information of an Alluxio path for display purpose.

Specified by:

getDisplayMountPointInfo in interface FileSystemMaster

Parameters:

path - an Alluxio path which must be a mount point

Returns:

the mount point information

Throws:

InvalidPathException

delete

public void delete(AlluxioURI path,
                   DeleteContext context)
            throws IOException,
                   FileDoesNotExistException,
                   DirectoryNotEmptyException,
                   InvalidPathException,
                   AccessControlException

Description copied from interface: FileSystemMaster

Deletes a given path.

This operation requires user to have WRITE permission on the parent of the path.

Specified by:

delete in interface FileSystemMaster

Parameters:

path - the path to delete

context - method context

Throws:

FileDoesNotExistException - if the file does not exist

DirectoryNotEmptyException - if recursive is false and the file is a nonempty directory

InvalidPathException - if the path is invalid

AccessControlException - if permission checking fails

IOException

deleteInternal

public void deleteInternal(RpcContext rpcContext,
                           LockedInodePath inodePath,
                           DeleteContext deleteContext,
                           boolean bypassPermCheck)
                    throws FileDoesNotExistException,
                           IOException,
                           DirectoryNotEmptyException,
                           InvalidPathException

Implements file deletion.

This method does not delete blocks. Instead, it returns deleted inodes so that their blocks can be deleted after the inode deletion journal entry has been written. We cannot delete blocks earlier because the inode deletion may fail, leaving us with inode containing deleted blocks. This method is used at: (1) delete() (2) unmount() (3) metadata sync (when a file/dir has been removed in UFS) Permission check should be skipped in (2) and (3).

Parameters:

rpcContext - the rpc context

inodePath - the file LockedInodePath

deleteContext - the method optitions

bypassPermCheck - whether the permission check has been done before entering this call

Throws:

FileDoesNotExistException

IOException

DirectoryNotEmptyException

InvalidPathException

getFileBlockInfoList

public List<FileBlockInfo> getFileBlockInfoList(AlluxioURI path)
                                         throws FileDoesNotExistException,
                                                InvalidPathException,
                                                AccessControlException,
                                                UnavailableException

Description copied from interface: FileSystemMaster

Gets the FileBlockInfo for all blocks of a file. If path is a directory, an exception is thrown.

This operation requires the client user to have READ permission on the the path.

Specified by:

getFileBlockInfoList in interface FileSystemMaster

Parameters:

path - the path to get the info for

Returns:

a list of FileBlockInfo for all the blocks of the given path

Throws:

FileDoesNotExistException - if the file does not exist or path is a directory

InvalidPathException - if the path of the given file is invalid

AccessControlException - if permission checking fails

UnavailableException

getInAlluxioFiles

public List<AlluxioURI> getInAlluxioFiles()
                                   throws UnavailableException

Specified by:

getInAlluxioFiles in interface FileSystemMaster

Returns:

absolute paths of all in-Alluxio files

Throws:

UnavailableException

getInMemoryFiles

public List<AlluxioURI> getInMemoryFiles()
                                  throws UnavailableException

Specified by:

getInMemoryFiles in interface FileSystemMaster

Returns:

absolute paths of all in-memory files

Throws:

UnavailableException

createDirectory

public long createDirectory(AlluxioURI path,
                            CreateDirectoryContext context)
                     throws InvalidPathException,
                            FileAlreadyExistsException,
                            IOException,
                            AccessControlException,
                            FileDoesNotExistException

Description copied from interface: FileSystemMaster

Creates a directory for a given path.

This operation requires the client user to have WRITE permission on the parent of the path.

Specified by:

createDirectory in interface FileSystemMaster

Parameters:

path - the path of the directory

context - method context

Returns:

the id of the created directory

Throws:

InvalidPathException - when the path is invalid

FileAlreadyExistsException - when there is already a file at path

AccessControlException - if permission checking fails

FileDoesNotExistException - if the parent of the path does not exist and the recursive option is false

IOException

rename

public void rename(AlluxioURI srcPath,
                   AlluxioURI dstPath,
                   RenameContext context)
            throws FileAlreadyExistsException,
                   FileDoesNotExistException,
                   InvalidPathException,
                   IOException,
                   AccessControlException

Description copied from interface: FileSystemMaster

Renames a file to a destination.

This operation requires users to have WRITE permission on the parent of the src path, and WRITE permission on the parent of the dst path.

Specified by:

rename in interface FileSystemMaster

Parameters:

srcPath - the source path to rename

dstPath - the destination path to rename the file to

context - method context

Throws:

FileAlreadyExistsException - if the file already exists

FileDoesNotExistException - if a non-existent file is encountered

InvalidPathException - if an invalid path is encountered

AccessControlException - if permission checking fails

IOException

free

public void free(AlluxioURI path,
                 FreeContext context)
          throws FileDoesNotExistException,
                 InvalidPathException,
                 AccessControlException,
                 UnexpectedAlluxioException,
                 IOException

Description copied from interface: FileSystemMaster

Frees or evicts all of the blocks of the file from alluxio storage. If the given file is a directory, and the 'recursive' flag is enabled, all descendant files will also be freed.

This operation requires users to have READ permission on the path.

Specified by:

free in interface FileSystemMaster

Parameters:

path - the path to free method

context - context to free method

Throws:

FileDoesNotExistException - if the file does not exist

InvalidPathException - if the given path is invalid

AccessControlException - if permission checking fails

UnexpectedAlluxioException - if the file or directory can not be freed

IOException

getPath

public AlluxioURI getPath(long fileId)
                   throws FileDoesNotExistException

Description copied from interface: FileSystemMaster

Gets the path of a file with the given id.

Specified by:

getPath in interface FileSystemMaster

Parameters:

fileId - the id of the file to look up

Returns:

the path of the file

Throws:

FileDoesNotExistException - raise if the file does not exist

getPinIdList

public Set<Long> getPinIdList()

Specified by:

getPinIdList in interface FileSystemMaster

Returns:

the set of inode ids which are pinned

getUfsAddress

public String getUfsAddress()

Specified by:

getUfsAddress in interface FileSystemMaster

Returns:

the ufs address for this master

getUfsInfo

public UfsInfo getUfsInfo(long mountId)

Specified by:

getUfsInfo in interface FileSystemMaster

Parameters:

mountId - the mount id to query

Returns:

the ufs information for the given mount id

getWhiteList

public List<String> getWhiteList()

Specified by:

getWhiteList in interface FileSystemMaster

Returns:

the white list

getLostFiles

public Set<Long> getLostFiles()

Specified by:

getLostFiles in interface FileSystemMaster

Returns:

all the files lost on the workers

getUfsManager

public MasterUfsManager getUfsManager()

Returns:

the MasterUfsManager for this master

loadMetadataIfNotExist

protected void loadMetadataIfNotExist(RpcContext rpcContext,
                                      AlluxioURI path,
                                      LoadMetadataContext context)
                               throws InvalidPathException,
                                      AccessControlException

Loads metadata for the path if it is (non-existing || load direct children is set). See shouldLoadMetadataIfNotExists(LockedInodePath, LoadMetadataContext).

Parameters:

rpcContext - the rpc context

path - the path to load metadata for

context - the LoadMetadataContext

Throws:

InvalidPathException

AccessControlException

updateMount

public void updateMount(AlluxioURI alluxioPath,
                        MountContext context)
                 throws FileAlreadyExistsException,
                        FileDoesNotExistException,
                        InvalidPathException,
                        IOException,
                        AccessControlException

Description copied from interface: FileSystemMaster

Update properties of an Alluxio mount point.

This operation requires users to have WRITE permission on the parent of the Alluxio path.

Specified by:

updateMount in interface FileSystemMaster

Parameters:

alluxioPath - the Alluxio path to update, must be a mount point

context - the mount context

Throws:

FileDoesNotExistException - if the given path does not exist

InvalidPathException - if the given path is not a mount point

AccessControlException - if the permission check fails

FileAlreadyExistsException

IOException

mount

public void mount(AlluxioURI alluxioPath,
                  AlluxioURI ufsPath,
                  MountContext context)
           throws FileAlreadyExistsException,
                  FileDoesNotExistException,
                  InvalidPathException,
                  IOException,
                  AccessControlException

Description copied from interface: FileSystemMaster

Mounts a UFS path onto an Alluxio path.

This operation requires users to have WRITE permission on the parent of the Alluxio path.

Specified by:

mount in interface FileSystemMaster

Parameters:

alluxioPath - the Alluxio path to mount to

ufsPath - the UFS path to mount

context - the mount context

Throws:

FileAlreadyExistsException - if the path to be mounted to already exists

FileDoesNotExistException - if the parent of the path to be mounted to does not exist

InvalidPathException - if an invalid path is encountered

AccessControlException - if the permission check fails

IOException

unmount

public void unmount(AlluxioURI alluxioPath)
             throws FileDoesNotExistException,
                    InvalidPathException,
                    IOException,
                    AccessControlException

Description copied from interface: FileSystemMaster

Unmounts a UFS path previously mounted onto an Alluxio path.

This operation requires users to have WRITE permission on the parent of the Alluxio path.

Specified by:

unmount in interface FileSystemMaster

Parameters:

alluxioPath - the Alluxio path to unmount, must be a mount point

Throws:

FileDoesNotExistException - if the path to be mounted does not exist

InvalidPathException - if the given path is not a mount point

AccessControlException - if the permission check fails

IOException

setAcl

public void setAcl(AlluxioURI path,
                   SetAclAction action,
                   List<AclEntry> entries,
                   SetAclContext context)
            throws FileDoesNotExistException,
                   AccessControlException,
                   InvalidPathException,
                   IOException

Description copied from interface: FileSystemMaster

Sets the ACL for a path.

Specified by:

setAcl in interface FileSystemMaster

Parameters:

path - the path to set attribute for

action - the set action to perform

entries - the list of ACL entries for setting ACL

context - the context for setting ACL

Throws:

FileDoesNotExistException - if the file does not exist

AccessControlException - if permission checking fails

InvalidPathException - if the given path is invalid

IOException

setAttribute

public void setAttribute(AlluxioURI path,
                         SetAttributeContext context)
                  throws FileDoesNotExistException,
                         AccessControlException,
                         InvalidPathException,
                         IOException

Description copied from interface: FileSystemMaster

Sets the file attribute.

This operation requires users to have WRITE permission on the path. In addition, the client user must be a super user when setting the owner, and must be a super user or the owner when setting the group or permission.

Specified by:

setAttribute in interface FileSystemMaster

Parameters:

path - the path to set attribute for

context - master operation context

Throws:

FileDoesNotExistException - if the file does not exist

AccessControlException - if permission checking fails

InvalidPathException - if the given path is invalid

IOException

scheduleAsyncPersistence

public void scheduleAsyncPersistence(AlluxioURI path,
                                     ScheduleAsyncPersistenceContext context)
                              throws AlluxioException,
                                     UnavailableException

Description copied from interface: FileSystemMaster

Schedules a file for async persistence.

Specified by:

scheduleAsyncPersistence in interface FileSystemMaster

Parameters:

path - the path of the file for persistence

context - the schedule async persistence context

Throws:

AlluxioException

UnavailableException

workerHeartbeat

public FileSystemCommand workerHeartbeat(long workerId,
                                         List<Long> persistedFiles,
                                         WorkerHeartbeatContext context)
                                  throws IOException

Description copied from interface: FileSystemMaster

Instructs a worker to persist the files.

Needs WRITE permission on the list of files.

Specified by:

workerHeartbeat in interface FileSystemMaster

Parameters:

workerId - the id of the worker that heartbeats

persistedFiles - the files that persisted on the worker

context - the method context

Returns:

the command for persisting the blocks of a file

Throws:

IOException

setAttributeSingleFile

protected void setAttributeSingleFile(RpcContext rpcContext,
                                      LockedInodePath inodePath,
                                      boolean updateUfs,
                                      long opTimeMs,
                                      SetAttributeContext context)
                               throws FileDoesNotExistException,
                                      InvalidPathException,
                                      AccessControlException

Parameters:

inodePath - the LockedInodePath to use

updateUfs - whether to update the UFS with the attribute change

opTimeMs - the operation time (in milliseconds)

context - the method context

Throws:

FileDoesNotExistException

InvalidPathException

AccessControlException

getWorkerInfoList

public List<WorkerInfo> getWorkerInfoList()
                                   throws UnavailableException

Specified by:

getWorkerInfoList in interface FileSystemMaster

Returns:

a list of WorkerInfo objects representing the workers in Alluxio

Throws:

UnavailableException

getLostWorkerList

public List<WorkerInfo> getLostWorkerList()
                                   throws UnavailableException

Specified by:

getLostWorkerList in interface FileSystemMaster

Returns:

a list of WorkerInfo objects representing the lost workers in Alluxio

Throws:

UnavailableException

getInodeCount

public long getInodeCount()

Description copied from interface: FileSystemMaster

Get the total inode count.

Specified by:

getInodeCount in interface FileSystemMaster

Returns:

inode count

updateUfsMode

public void updateUfsMode(AlluxioURI ufsPath,
                          UfsMode ufsMode)
                   throws InvalidPathException,
                          InvalidArgumentException,
                          UnavailableException,
                          AccessControlException

Description copied from interface: FileSystemMaster

Update the operation mode for the given ufs path under one or more mount points.

Specified by:

updateUfsMode in interface FileSystemMaster

Parameters:

ufsPath - the physical ufs path

ufsMode - the ufs operation mode

Throws:

InvalidPathException - if ufs path is not used by any mount point

InvalidArgumentException - if arguments for the method are invalid

UnavailableException

AccessControlException

createAuditContext

protected FileSystemMasterAuditContext createAuditContext(String command,
                                                          AlluxioURI srcPath,
                                                          @Nullable
                                                          AlluxioURI dstPath,
                                                          @Nullable
                                                          Inode srcInode)

Creates a FileSystemMasterAuditContext instance.

Parameters:

command - the command to be logged by this AuditContext

srcPath - the source path of this command

dstPath - the destination path of this command

srcInode - the source inode of this command

Returns:

newly-created FileSystemMasterAuditContext instance

createBlockDeletionContext

protected BlockDeletionContext createBlockDeletionContext()

createRpcContext

public RpcContext createRpcContext()
                            throws UnavailableException

Returns:

a context for executing an RPC

Throws:

UnavailableException

createRpcContext

public RpcContext createRpcContext(OperationContext operationContext)
                            throws UnavailableException

Parameters:

operationContext - the operation context

Returns:

a context for executing an RPC

Throws:

UnavailableException

createSyncLockingScheme

protected LockingScheme createSyncLockingScheme(AlluxioURI path,
                                                FileSystemMasterCommonPOptions options,
                                                DescendantType descendantType)
                                         throws InvalidPathException

Throws:

InvalidPathException

getTimeSeries

public List<TimeSeries> getTimeSeries()

Specified by:

getTimeSeries in interface FileSystemMaster

Returns:

the time series data stored by the master

reverseResolve

public AlluxioURI reverseResolve(AlluxioURI ufsUri)
                          throws InvalidPathException

Description copied from interface: FileSystemMaster

Reverse path resolve a ufs uri to an Alluxio path.

Specified by:

reverseResolve in interface FileSystemMaster

Parameters:

ufsUri - ufs uri

Returns:

alluxio path

Throws:

InvalidPathException

getRootInodeOwner

@Nullable
public String getRootInodeOwner()

Specified by:

getRootInodeOwner in interface FileSystemMaster

Returns:

the owner of the root inode, null if the inode tree is not initialized

getStateLockSharedWaitersAndHolders

public Collection<String> getStateLockSharedWaitersAndHolders()

Specified by:

getStateLockSharedWaitersAndHolders in interface FileSystemMaster

Returns:

the list of thread identifiers that are waiting and holding the state lock

getSyncPathCache

public UfsSyncPathCache getSyncPathCache()

Returns:

the invalidation sync cache

getMountTable

public MountTable getMountTable()

Returns:

the mount table

needsSync

public void needsSync(AlluxioURI path)
               throws InvalidPathException

Description copied from interface: FileSystemMaster

Mark a path as needed synchronization with the UFS, when this path or any of its children are accessed, a sync with the UFS will be performed.

Specified by:

needsSync in interface FileSystemMaster

Parameters:

path - the path to invalidate

Throws:

InvalidPathException

getScheduler

public Scheduler getScheduler()

Get scheduler.

Specified by:

getScheduler in interface FileSystemMaster

Returns:

scheduler