alluxio.master.file

Interface FileSystemMaster

All Superinterfaces:

Checkpointed, Journaled, JournalEntryIterable, Master, Server<Boolean>

All Known Implementing Classes:

DefaultFileSystemMaster

public interface FileSystemMaster
extends Master

The interface of file system master service.

Method Summary

Modifier and Type	Method and Description
void	activeSyncMetadata(AlluxioURI path, Collection<AlluxioURI> changedFiles, ExecutorService executorService) Starts a batch sync with a list of changed files passed in.
void	checkAccess(AlluxioURI path, CheckAccessContext context) Checks access to path.
List<AlluxioURI>	checkConsistency(AlluxioURI path, CheckConsistencyContext context) Checks the consistency of the files and directories in the subtree under the path.
void	cleanupUfs() Periodically clean up the under file systems.
void	completeFile(AlluxioURI path, CompleteFileContext context) Completes a file.
long	createDirectory(AlluxioURI path, CreateDirectoryContext context) Creates a directory for a given path.
FileInfo	createFile(AlluxioURI path, CreateFileContext context) Creates a file (not a directory) for a given path.
void	delete(AlluxioURI path, DeleteContext context) Deletes a given path.
void	free(AlluxioURI path, FreeContext context) Frees or evicts all of the blocks of the file from alluxio storage.
MountPointInfo	getDisplayMountPointInfo(AlluxioURI path) Gets the mount point information of an Alluxio path for display purpose.
List<FileBlockInfo>	getFileBlockInfoList(AlluxioURI path) Gets the FileBlockInfo for all blocks of a file.
long	getFileId(AlluxioURI path) Returns the file id for a given path.
FileInfo	getFileInfo(AlluxioURI path, GetStatusContext context) Returns the FileInfo for a given path.
FileInfo	getFileInfo(long fileId) Returns the FileInfo for a given file id.
FileSystemMasterView	getFileSystemMasterView()
List<AlluxioURI>	getInAlluxioFiles()
List<AlluxioURI>	getInMemoryFiles()
long	getInodeCount()
List<Long>	getLostFiles()
Map<String,MountPointInfo>	getMountPointInfoSummary()
long	getNewBlockIdForFile(AlluxioURI path) Gets a new block id for the next block of a given file to write to.
int	getNumberOfPinnedFiles()
AlluxioURI	getPath(long fileId) Gets the path of a file with the given id.
PersistenceState	getPersistenceState(long fileId) Returns the persistence state for a file id.
Set<Long>	getPinIdList()
String	getRootInodeOwner()
List<String>	getStateLockSharedWaitersAndHolders()
List<SyncPointInfo>	getSyncPathList()
List<TimeSeries>	getTimeSeries()
String	getUfsAddress()
UfsInfo	getUfsInfo(long mountId)
List<String>	getWhiteList()
List<WorkerInfo>	getWorkerInfoList()
List<FileInfo>	listStatus(AlluxioURI path, ListStatusContext context) Returns a list of FileInfo for a given path.
void	listStatus(AlluxioURI path, ListStatusContext context, ResultStream<FileInfo> resultStream) Enumerates given path to given batch tracker.
void	mount(AlluxioURI alluxioPath, AlluxioURI ufsPath, MountContext context) Mounts a UFS path onto an Alluxio path.
boolean	recordActiveSyncTxid(long txId, long mountId) Journal the active sync transaction id so that we can restart more efficiently.
void	rename(AlluxioURI srcPath, AlluxioURI dstPath, RenameContext context) Renames a file to a destination.
AlluxioURI	reverseResolve(AlluxioURI ufsUri) Reverse path resolve a ufs uri to an Alluxio path.
void	scheduleAsyncPersistence(AlluxioURI path, ScheduleAsyncPersistenceContext context) Schedules a file for async persistence.
void	setAcl(AlluxioURI path, SetAclAction action, List<AclEntry> entries, SetAclContext context) Sets the ACL for a path.
void	setAttribute(AlluxioURI path, SetAttributeContext options) Sets the file attribute.
void	startSync(AlluxioURI alluxioURI) starts active sync on a specified alluxioURI.
void	stopSync(AlluxioURI alluxioURI) stops active sync on a specific syncpoint.
void	unmount(AlluxioURI alluxioPath) Unmounts a UFS path previously mounted onto an Alluxio path.
void	updateMount(AlluxioURI alluxioPath, MountContext context) Update properties of an Alluxio mount point.
void	updateUfsMode(AlluxioURI ufsPath, UfsMode ufsMode) Update the operation mode for the given ufs path under one or more mount points.
void	validateInodeBlocks(boolean repair) Checks the integrity of the inodes with respect to the blocks of the system.
FileSystemCommand	workerHeartbeat(long workerId, List<Long> persistedFiles, WorkerHeartbeatContext context) Instructs a worker to persist the files.

Methods inherited from interface alluxio.master.Master

createJournalContext, getMasterContext

Methods inherited from interface alluxio.master.journal.Journaled

applyAndJournal, processJournalEntry, resetState, restoreFromCheckpoint, writeToCheckpoint

Methods inherited from interface alluxio.master.journal.checkpoint.Checkpointed

getCheckpointName

Methods inherited from interface alluxio.master.journal.JournalEntryIterable

getJournalEntryIterator

Methods inherited from interface alluxio.Server

close, getDependencies, getName, getServices, start, stop

Method Detail

cleanupUfs

void cleanupUfs()

Periodically clean up the under file systems.

getFileId

long getFileId(AlluxioURI path)
        throws AccessControlException,
               UnavailableException

Returns the file id for a given path. If the given path does not exist in Alluxio, the method attempts to load it from UFS.

This operation requires users to have READ permission of the path.

Parameters:

path - the path to get the file id for

Returns:

the file id for a given path, or -1 if there is no file at that path

Throws:

AccessControlException - if permission checking fails

UnavailableException

getFileInfo

FileInfo getFileInfo(long fileId)
              throws FileDoesNotExistException,
                     AccessControlException,
                     UnavailableException

Returns the FileInfo for a given file id. This method is not user-facing but supposed to be called by other internal servers (e.g., block workers, web UI).

Parameters:

fileId - the file id to get the FileInfo for

Returns:

the FileInfo for the given file

Throws:

FileDoesNotExistException - if the file does not exist

AccessControlException - if permission denied

UnavailableException

getFileInfo

FileInfo getFileInfo(AlluxioURI path,
                     GetStatusContext context)
              throws FileDoesNotExistException,
                     InvalidPathException,
                     AccessControlException,
                     UnavailableException,
                     IOException

Returns the FileInfo for a given path.

This operation requires users to have READ permission on the path.

Parameters:

path - the path to get the FileInfo for

context - the method context

Returns:

the FileInfo for the given file id

Throws:

FileDoesNotExistException - if the file does not exist

InvalidPathException - if the file path is not valid

AccessControlException - if permission checking fails

UnavailableException

IOException

getPersistenceState

PersistenceState getPersistenceState(long fileId)
                              throws FileDoesNotExistException

Returns the persistence state for a file id.

Parameters:

fileId - the file id

Returns:

the PersistenceState for the given file id

Throws:

FileDoesNotExistException - if the file does not exist

listStatus

List<FileInfo> listStatus(AlluxioURI path,
                          ListStatusContext context)
                   throws AccessControlException,
                          FileDoesNotExistException,
                          InvalidPathException,
                          UnavailableException,
                          IOException

Returns a list of FileInfo for a given path. If the given path is a file, the list only contains a single object. If it is a directory, the resulting list contains all direct children of the directory.

This operation requires users to have READ permission on the path, and also EXECUTE permission on the path if it is a directory.

Parameters:

path - the path to get the FileInfo list for

context - the method context

Returns:

the list of FileInfos

Throws:

AccessControlException - if permission checking fails

FileDoesNotExistException - if the file does not exist

InvalidPathException - if the path is invalid

UnavailableException

IOException

listStatus

void listStatus(AlluxioURI path,
                ListStatusContext context,
                ResultStream<FileInfo> resultStream)
         throws AccessControlException,
                FileDoesNotExistException,
                InvalidPathException,
                UnavailableException,
                IOException

Enumerates given path to given batch tracker. If the given path is a file, the list only contains a single object. If it is a directory, the resulting list contains all direct children of the directory.

This operation requires users to have READ permission on the path, and also EXECUTE permission on the path if it is a directory.

Parameters:

path - the path to get the FileInfo list for

context - the method context

resultStream - the stream to receive individual results

Throws:

AccessControlException - if permission checking fails

FileDoesNotExistException - if the file does not exist

InvalidPathException - if the path is invalid

UnavailableException

IOException

getFileSystemMasterView

FileSystemMasterView getFileSystemMasterView()

Returns:

a read-only view of the file system master

checkAccess

void checkAccess(AlluxioURI path,
                 CheckAccessContext context)
          throws FileDoesNotExistException,
                 InvalidPathException,
                 AccessControlException,
                 IOException

Checks access to path.

Parameters:

path - the path to check access to

context - the method context

Throws:

FileDoesNotExistException - if the file does not exist

AccessControlException - if permission checking fails

InvalidPathException - if the given path is invalid

IOException

checkConsistency

List<AlluxioURI> checkConsistency(AlluxioURI path,
                                  CheckConsistencyContext context)
                           throws AccessControlException,
                                  FileDoesNotExistException,
                                  InvalidPathException,
                                  IOException

Checks the consistency of the files and directories in the subtree under the path.

Parameters:

path - the root of the subtree to check

context - the context to use for the checkConsistency method

Returns:

a list of paths in Alluxio which are not consistent with the under storage

Throws:

AccessControlException - if the permission checking fails

FileDoesNotExistException - if the path does not exist

InvalidPathException - if the path is invalid

IOException

completeFile

void completeFile(AlluxioURI path,
                  CompleteFileContext context)
           throws BlockInfoException,
                  FileDoesNotExistException,
                  InvalidPathException,
                  InvalidFileSizeException,
                  FileAlreadyCompletedException,
                  AccessControlException,
                  UnavailableException

Completes a file. After a file is completed, it cannot be written to.

This operation requires users to have WRITE permission on the path.

Parameters:

path - the file path to complete

context - the method context

Throws:

BlockInfoException - if a block information exception is encountered

FileDoesNotExistException - if the file does not exist

InvalidPathException - if an invalid path is encountered

InvalidFileSizeException - if an invalid file size is encountered

FileAlreadyCompletedException - if the file is already completed

AccessControlException - if permission checking fails

UnavailableException

createFile

FileInfo createFile(AlluxioURI path,
                    CreateFileContext context)
             throws AccessControlException,
                    InvalidPathException,
                    FileAlreadyExistsException,
                    BlockInfoException,
                    IOException,
                    FileDoesNotExistException

Creates a file (not a directory) for a given path.

This operation requires WRITE permission on the parent of this path.

Parameters:

path - the file to create

context - the method context

Returns:

the file info of the created file

Throws:

InvalidPathException - if an invalid path is encountered

FileAlreadyExistsException - if the file already exists

BlockInfoException - if an invalid block information is encountered

AccessControlException - if permission checking fails

FileDoesNotExistException - if the parent of the path does not exist and the recursive option is false

IOException

getNewBlockIdForFile

long getNewBlockIdForFile(AlluxioURI path)
                   throws FileDoesNotExistException,
                          InvalidPathException,
                          AccessControlException,
                          UnavailableException

Gets a new block id for the next block of a given file to write to.

This operation requires users to have WRITE permission on the path as this API is called when creating a new block for a file.

Parameters:

path - the path of the file to get the next block id for

Returns:

the next block id for the given file

Throws:

FileDoesNotExistException - if the file does not exist

InvalidPathException - if the given path is not valid

AccessControlException - if permission checking fails

UnavailableException

getMountPointInfoSummary

Map<String,MountPointInfo> getMountPointInfoSummary()

Returns:

a snapshot of the mount table as a mapping of Alluxio path to MountPointInfo

getDisplayMountPointInfo

MountPointInfo getDisplayMountPointInfo(AlluxioURI path)
                                 throws InvalidPathException

Gets the mount point information of an Alluxio path for display purpose.

Parameters:

path - an Alluxio path which must be a mount point

Returns:

the mount point information

Throws:

InvalidPathException

getInodeCount

long getInodeCount()

Returns:

the number of files and directories

getNumberOfPinnedFiles

int getNumberOfPinnedFiles()

Returns:

the number of pinned files and directories

delete

void delete(AlluxioURI path,
            DeleteContext context)
     throws IOException,
            FileDoesNotExistException,
            DirectoryNotEmptyException,
            InvalidPathException,
            AccessControlException

Deletes a given path.

This operation requires user to have WRITE permission on the parent of the path.

Parameters:

path - the path to delete

context - method context

Throws:

DirectoryNotEmptyException - if recursive is false and the file is a nonempty directory

FileDoesNotExistException - if the file does not exist

AccessControlException - if permission checking fails

InvalidPathException - if the path is invalid

IOException

getFileBlockInfoList

List<FileBlockInfo> getFileBlockInfoList(AlluxioURI path)
                                  throws FileDoesNotExistException,
                                         InvalidPathException,
                                         AccessControlException,
                                         UnavailableException

Gets the FileBlockInfo for all blocks of a file. If path is a directory, an exception is thrown.

This operation requires the client user to have READ permission on the the path.

Parameters:

path - the path to get the info for

Returns:

a list of FileBlockInfo for all the blocks of the given path

Throws:

FileDoesNotExistException - if the file does not exist or path is a directory

InvalidPathException - if the path of the given file is invalid

AccessControlException - if permission checking fails

UnavailableException

getInAlluxioFiles

List<AlluxioURI> getInAlluxioFiles()
                            throws UnavailableException

Returns:

absolute paths of all in-Alluxio files

Throws:

UnavailableException

getInMemoryFiles

List<AlluxioURI> getInMemoryFiles()
                           throws UnavailableException

Returns:

absolute paths of all in-memory files

Throws:

UnavailableException

createDirectory

long createDirectory(AlluxioURI path,
                     CreateDirectoryContext context)
              throws InvalidPathException,
                     FileAlreadyExistsException,
                     IOException,
                     AccessControlException,
                     FileDoesNotExistException

Creates a directory for a given path.

This operation requires the client user to have WRITE permission on the parent of the path.

Parameters:

path - the path of the directory

context - method context

Returns:

the id of the created directory

Throws:

InvalidPathException - when the path is invalid

FileAlreadyExistsException - when there is already a file at path

AccessControlException - if permission checking fails

FileDoesNotExistException - if the parent of the path does not exist and the recursive option is false

IOException

rename

void rename(AlluxioURI srcPath,
            AlluxioURI dstPath,
            RenameContext context)
     throws FileAlreadyExistsException,
            FileDoesNotExistException,
            InvalidPathException,
            IOException,
            AccessControlException

Renames a file to a destination.

This operation requires users to have WRITE permission on the parent of the src path, and WRITE permission on the parent of the dst path.

Parameters:

srcPath - the source path to rename

dstPath - the destination path to rename the file to

context - method context

Throws:

FileDoesNotExistException - if a non-existent file is encountered

InvalidPathException - if an invalid path is encountered

AccessControlException - if permission checking fails

FileAlreadyExistsException - if the file already exists

IOException

free

void free(AlluxioURI path,
          FreeContext context)
   throws FileDoesNotExistException,
          InvalidPathException,
          AccessControlException,
          UnexpectedAlluxioException,
          IOException

Frees or evicts all of the blocks of the file from alluxio storage. If the given file is a directory, and the 'recursive' flag is enabled, all descendant files will also be freed.

This operation requires users to have READ permission on the path.

Parameters:

path - the path to free method

context - context to free method

Throws:

FileDoesNotExistException - if the file does not exist

AccessControlException - if permission checking fails

InvalidPathException - if the given path is invalid

UnexpectedAlluxioException - if the file or directory can not be freed

IOException

getPath

AlluxioURI getPath(long fileId)
            throws FileDoesNotExistException

Gets the path of a file with the given id.

Parameters:

fileId - the id of the file to look up

Returns:

the path of the file

Throws:

FileDoesNotExistException - raise if the file does not exist

getPinIdList

Set<Long> getPinIdList()

Returns:

the set of inode ids which are pinned

getUfsAddress

String getUfsAddress()

Returns:

the ufs address for this master

getUfsInfo

UfsInfo getUfsInfo(long mountId)

Parameters:

mountId - the mount id to query

Returns:

the ufs information for the given mount id

getWhiteList

List<String> getWhiteList()

Returns:

the white list

getLostFiles

List<Long> getLostFiles()

Returns:

all the files lost on the workers

mount

void mount(AlluxioURI alluxioPath,
           AlluxioURI ufsPath,
           MountContext context)
    throws FileAlreadyExistsException,
           FileDoesNotExistException,
           InvalidPathException,
           IOException,
           AccessControlException

Mounts a UFS path onto an Alluxio path.

This operation requires users to have WRITE permission on the parent of the Alluxio path.

Parameters:

alluxioPath - the Alluxio path to mount to

ufsPath - the UFS path to mount

context - the mount context

Throws:

FileAlreadyExistsException - if the path to be mounted to already exists

FileDoesNotExistException - if the parent of the path to be mounted to does not exist

InvalidPathException - if an invalid path is encountered

AccessControlException - if the permission check fails

IOException

unmount

void unmount(AlluxioURI alluxioPath)
      throws FileDoesNotExistException,
             InvalidPathException,
             IOException,
             AccessControlException

Unmounts a UFS path previously mounted onto an Alluxio path.

This operation requires users to have WRITE permission on the parent of the Alluxio path.

Parameters:

alluxioPath - the Alluxio path to unmount, must be a mount point

Throws:

FileDoesNotExistException - if the path to be mounted does not exist

InvalidPathException - if the given path is not a mount point

AccessControlException - if the permission check fails

IOException

updateMount

void updateMount(AlluxioURI alluxioPath,
                 MountContext context)
          throws FileAlreadyExistsException,
                 FileDoesNotExistException,
                 InvalidPathException,
                 IOException,
                 AccessControlException

Update properties of an Alluxio mount point.

This operation requires users to have WRITE permission on the parent of the Alluxio path.

Parameters:

alluxioPath - the Alluxio path to update, must be a mount point

context - the mount context

Throws:

FileDoesNotExistException - if the given path does not exist

InvalidPathException - if the given path is not a mount point

AccessControlException - if the permission check fails

FileAlreadyExistsException

IOException

setAcl

void setAcl(AlluxioURI path,
            SetAclAction action,
            List<AclEntry> entries,
            SetAclContext context)
     throws FileDoesNotExistException,
            AccessControlException,
            InvalidPathException,
            IOException

Sets the ACL for a path.

Parameters:

path - the path to set attribute for

action - the set action to perform

entries - the list of ACL entries for setting ACL

context - the context for setting ACL

Throws:

FileDoesNotExistException - if the file does not exist

AccessControlException - if permission checking fails

InvalidPathException - if the given path is invalid

IOException

setAttribute

void setAttribute(AlluxioURI path,
                  SetAttributeContext options)
           throws FileDoesNotExistException,
                  AccessControlException,
                  InvalidPathException,
                  IOException

Sets the file attribute.

This operation requires users to have WRITE permission on the path. In addition, the client user must be a super user when setting the owner, and must be a super user or the owner when setting the group or permission.

Parameters:

path - the path to set attribute for

options - master operation context

Throws:

FileDoesNotExistException - if the file does not exist

AccessControlException - if permission checking fails

InvalidPathException - if the given path is invalid

IOException

scheduleAsyncPersistence

void scheduleAsyncPersistence(AlluxioURI path,
                              ScheduleAsyncPersistenceContext context)
                       throws AlluxioException,
                              UnavailableException

Schedules a file for async persistence.

Parameters:

path - the path of the file for persistence

context - the schedule async persistence context

Throws:

AlluxioException

UnavailableException

updateUfsMode

void updateUfsMode(AlluxioURI ufsPath,
                   UfsMode ufsMode)
            throws InvalidPathException,
                   InvalidArgumentException,
                   UnavailableException,
                   AccessControlException

Update the operation mode for the given ufs path under one or more mount points.

Parameters:

ufsPath - the physical ufs path

ufsMode - the ufs operation mode

Throws:

InvalidPathException - if ufs path is not used by any mount point

InvalidArgumentException - if arguments for the method are invalid

UnavailableException

AccessControlException

validateInodeBlocks

void validateInodeBlocks(boolean repair)
                  throws UnavailableException

Checks the integrity of the inodes with respect to the blocks of the system.

Parameters:

repair - if true, will attempt to repair the state of the system when inconsistencies are discovered

Throws:

UnavailableException - if the repair attempt fails

workerHeartbeat

FileSystemCommand workerHeartbeat(long workerId,
                                  List<Long> persistedFiles,
                                  WorkerHeartbeatContext context)
                           throws FileDoesNotExistException,
                                  InvalidPathException,
                                  AccessControlException,
                                  IOException

Instructs a worker to persist the files.

Needs WRITE permission on the list of files.

Parameters:

workerId - the id of the worker that heartbeats

persistedFiles - the files that persisted on the worker

context - the method context

Returns:

the command for persisting the blocks of a file

Throws:

FileDoesNotExistException - if the file does not exist

InvalidPathException - if the file path corresponding to the file id is invalid

AccessControlException - if permission checking fails

IOException

getWorkerInfoList

List<WorkerInfo> getWorkerInfoList()
                            throws UnavailableException

Returns:

a list of WorkerInfo objects representing the workers in Alluxio

Throws:

UnavailableException

getSyncPathList

List<SyncPointInfo> getSyncPathList()
                             throws UnavailableException,
                                    AccessControlException

Returns:

the list of sync path

Throws:

UnavailableException

AccessControlException

startSync

void startSync(AlluxioURI alluxioURI)
        throws IOException,
               InvalidPathException,
               AccessControlException,
               ConnectionFailedException

starts active sync on a specified alluxioURI.

Parameters:

alluxioURI - sync point which is a valid path in Alluxio namespace

Throws:

UnavailableException

InvalidPathException

AccessControlException

IOException

ConnectionFailedException

stopSync

void stopSync(AlluxioURI alluxioURI)
       throws IOException,
              InvalidPathException,
              AccessControlException

stops active sync on a specific syncpoint.

Parameters:

alluxioURI - alluxio path that has been added as a sync point

Throws:

UnavailableException

InvalidPathException

AccessControlException

IOException

activeSyncMetadata

void activeSyncMetadata(AlluxioURI path,
                        Collection<AlluxioURI> changedFiles,
                        ExecutorService executorService)
                 throws IOException

Starts a batch sync with a list of changed files passed in. If no files are passed in, sync the entire path.

Parameters:

path - the path to sync

changedFiles - collection of files that are changed under the path to sync

executorService - executor for executing parallel syncs

Throws:

IOException

recordActiveSyncTxid

boolean recordActiveSyncTxid(long txId,
                             long mountId)

Journal the active sync transaction id so that we can restart more efficiently.

Parameters:

txId - transaction id

mountId - mount id

Returns:

true if successfully recorded in the journal

getTimeSeries

List<TimeSeries> getTimeSeries()

Returns:

the time series data stored by the master

reverseResolve

AlluxioURI reverseResolve(AlluxioURI ufsUri)
                   throws InvalidPathException

Reverse path resolve a ufs uri to an Alluxio path.

Parameters:

ufsUri - ufs uri

Returns:

alluxio path

Throws:

InvalidPathException

getRootInodeOwner

String getRootInodeOwner()

Returns:

the owner of the root inode, null if the inode tree is not initialized

getStateLockSharedWaitersAndHolders

List<String> getStateLockSharedWaitersAndHolders()

Returns:

the list of thread identifiers that are waiting and holding the state lock