alluxio.master.file

Interface FileSystemMaster

All Superinterfaces:

JournalEntryIterable, JournalEntryStateMachine, Master, Server<Boolean>

All Known Implementing Classes:

DefaultFileSystemMaster

public interface FileSystemMaster
extends Master

The interface of file system master.

Method Summary

Modifier and Type	Method and Description
List<AlluxioURI>	checkConsistency(AlluxioURI path, CheckConsistencyOptions options) Checks the consistency of the files and directories in the subtree under the path.
void	completeFile(AlluxioURI path, CompleteFileOptions options) Completes a file.
long	createDirectory(AlluxioURI path, CreateDirectoryOptions options) Creates a directory for a given path.
long	createFile(AlluxioURI path, CreateFileOptions options) Creates a file (not a directory) for a given path.
void	delete(AlluxioURI path, DeleteOptions options) Deletes a given path.
void	free(AlluxioURI path, FreeOptions options) Frees or evicts all of the blocks of the file from alluxio storage.
MountPointInfo	getDisplayMountPointInfo(AlluxioURI path) Gets the mount point information of an Alluxio path for display purpose.
List<FileBlockInfo>	getFileBlockInfoList(AlluxioURI path) Gets the FileBlockInfo for all blocks of a file.
long	getFileId(AlluxioURI path) Returns the file id for a given path.
FileInfo	getFileInfo(AlluxioURI path, GetStatusOptions options) Returns the FileInfo for a given path.
FileInfo	getFileInfo(long fileId) Returns the FileInfo for a given file id.
FileSystemMasterView	getFileSystemMasterView()
List<AlluxioURI>	getInAlluxioFiles()
List<AlluxioURI>	getInMemoryFiles()
List<Long>	getLostFiles()
MountPointInfo	getMountPointInfo(AlluxioURI path) Deprecated. will be removed after 2.0
Map<String,MountPointInfo>	getMountTable()
long	getNewBlockIdForFile(AlluxioURI path) Gets a new block id for the next block of a given file to write to.
int	getNumberOfPaths()
int	getNumberOfPinnedFiles()
AlluxioURI	getPath(long fileId) Gets the path of a file with the given id.
PersistenceState	getPersistenceState(long fileId) Returns the persistence state for a file id.
Set<Long>	getPinIdList()
StartupConsistencyCheck	getStartupConsistencyCheck()
String	getUfsAddress()
UfsInfo	getUfsInfo(long mountId)
List<String>	getWhiteList()
List<WorkerInfo>	getWorkerInfoList()
List<FileInfo>	listStatus(AlluxioURI path, ListStatusOptions listStatusOptions) Returns a list of FileInfo for a given path.
long	loadMetadata(AlluxioURI path, LoadMetadataOptions options) Loads metadata for the object identified by the given path from UFS into Alluxio.
void	mount(AlluxioURI alluxioPath, AlluxioURI ufsPath, MountOptions options) Mounts a UFS path onto an Alluxio path.
long	reinitializeFile(AlluxioURI path, long blockSizeBytes, long ttl, TtlAction ttlAction) Reinitializes the blocks of an existing open file.
void	rename(AlluxioURI srcPath, AlluxioURI dstPath, RenameOptions options) Renames a file to a destination.
void	reportLostFile(long fileId) Reports a file as lost.
void	resetFile(long fileId) Resets a file.
void	scheduleAsyncPersistence(AlluxioURI path) Schedules a file for async persistence.
void	setAttribute(AlluxioURI path, SetAttributeOptions options) Sets the file attribute.
void	unmount(AlluxioURI alluxioPath) Unmounts a UFS path previously mounted onto an Alluxio path.
void	updateUfsMode(AlluxioURI ufsPath, UnderFileSystem.UfsMode ufsMode) Update the operation mode for the given ufs path under one or more mount points.
void	validateInodeBlocks(boolean repair) Checks the integrity of the inodes with respect to the blocks of the system.
FileSystemCommand	workerHeartbeat(long workerId, List<Long> persistedFiles, WorkerHeartbeatOptions options) Instructs a worker to persist the files.

Methods inherited from interface alluxio.master.Master

createJournalContext

Methods inherited from interface alluxio.master.journal.JournalEntryStateMachine

getName, processJournalEntry, resetState

Methods inherited from interface alluxio.master.journal.JournalEntryIterable

getJournalEntryIterator

Methods inherited from interface alluxio.Server

getDependencies, getName, getServices, start, stop

Method Detail

getStartupConsistencyCheck

StartupConsistencyCheck getStartupConsistencyCheck()

Returns:

the status of the startup consistency check and inconsistent paths if it is complete

getFileId

long getFileId(AlluxioURI path)
        throws AccessControlException,
               UnavailableException

Returns the file id for a given path. If the given path does not exist in Alluxio, the method attempts to load it from UFS.

This operation requires users to have READ permission of the path.

Parameters:

path - the path to get the file id for

Returns:

the file id for a given path, or -1 if there is no file at that path

Throws:

AccessControlException - if permission checking fails

UnavailableException

getFileInfo

FileInfo getFileInfo(long fileId)
              throws FileDoesNotExistException,
                     AccessControlException,
                     UnavailableException

Returns the FileInfo for a given file id. This method is not user-facing but supposed to be called by other internal servers (e.g., block workers, lineage master, web UI).

Parameters:

fileId - the file id to get the FileInfo for

Returns:

the FileInfo for the given file

Throws:

FileDoesNotExistException - if the file does not exist

AccessControlException - if permission denied

UnavailableException

getFileInfo

FileInfo getFileInfo(AlluxioURI path,
                     GetStatusOptions options)
              throws FileDoesNotExistException,
                     InvalidPathException,
                     AccessControlException,
                     UnavailableException,
                     IOException

Returns the FileInfo for a given path.

This operation requires users to have READ permission on the path.

Parameters:

path - the path to get the FileInfo for

options - the GetStatusOptions

Returns:

the FileInfo for the given file id

Throws:

FileDoesNotExistException - if the file does not exist

InvalidPathException - if the file path is not valid

AccessControlException - if permission checking fails

UnavailableException

IOException

getPersistenceState

PersistenceState getPersistenceState(long fileId)
                              throws FileDoesNotExistException

Returns the persistence state for a file id. This method is used by the lineage master.

Parameters:

fileId - the file id

Returns:

the PersistenceState for the given file id

Throws:

FileDoesNotExistException - if the file does not exist

listStatus

List<FileInfo> listStatus(AlluxioURI path,
                          ListStatusOptions listStatusOptions)
                   throws AccessControlException,
                          FileDoesNotExistException,
                          InvalidPathException,
                          UnavailableException,
                          IOException

Returns a list of FileInfo for a given path. If the given path is a file, the list only contains a single object. If it is a directory, the resulting list contains all direct children of the directory.

This operation requires users to have READ permission on the path, and also EXECUTE permission on the path if it is a directory.

Parameters:

path - the path to get the FileInfo list for

listStatusOptions - the ListStatusOptions

Returns:

the list of FileInfos

Throws:

AccessControlException - if permission checking fails

FileDoesNotExistException - if the file does not exist

InvalidPathException - if the path is invalid

UnavailableException

IOException

getFileSystemMasterView

FileSystemMasterView getFileSystemMasterView()

Returns:

a read-only view of the file system master

checkConsistency

List<AlluxioURI> checkConsistency(AlluxioURI path,
                                  CheckConsistencyOptions options)
                           throws AccessControlException,
                                  FileDoesNotExistException,
                                  InvalidPathException,
                                  IOException

Checks the consistency of the files and directories in the subtree under the path.

Parameters:

path - the root of the subtree to check

options - the options to use for the checkConsistency method

Returns:

a list of paths in Alluxio which are not consistent with the under storage

Throws:

AccessControlException - if the permission checking fails

FileDoesNotExistException - if the path does not exist

InvalidPathException - if the path is invalid

IOException

completeFile

void completeFile(AlluxioURI path,
                  CompleteFileOptions options)
           throws BlockInfoException,
                  FileDoesNotExistException,
                  InvalidPathException,
                  InvalidFileSizeException,
                  FileAlreadyCompletedException,
                  AccessControlException,
                  UnavailableException

Completes a file. After a file is completed, it cannot be written to.

This operation requires users to have WRITE permission on the path.

Parameters:

path - the file path to complete

options - the method options

Throws:

BlockInfoException - if a block information exception is encountered

FileDoesNotExistException - if the file does not exist

InvalidPathException - if an invalid path is encountered

InvalidFileSizeException - if an invalid file size is encountered

FileAlreadyCompletedException - if the file is already completed

AccessControlException - if permission checking fails

UnavailableException

createFile

long createFile(AlluxioURI path,
                CreateFileOptions options)
         throws AccessControlException,
                InvalidPathException,
                FileAlreadyExistsException,
                BlockInfoException,
                IOException,
                FileDoesNotExistException

Creates a file (not a directory) for a given path.

This operation requires WRITE permission on the parent of this path.

Parameters:

path - the file to create

options - method options

Returns:

the id of the created file

Throws:

InvalidPathException - if an invalid path is encountered

FileAlreadyExistsException - if the file already exists

BlockInfoException - if an invalid block information is encountered

AccessControlException - if permission checking fails

FileDoesNotExistException - if the parent of the path does not exist and the recursive option is false

IOException

reinitializeFile

long reinitializeFile(AlluxioURI path,
                      long blockSizeBytes,
                      long ttl,
                      TtlAction ttlAction)
               throws InvalidPathException,
                      FileDoesNotExistException,
                      UnavailableException

Reinitializes the blocks of an existing open file.

Parameters:

path - the path to the file

blockSizeBytes - the new block size

ttl - the ttl

ttlAction - action to take after Ttl expiry

Returns:

the file id

Throws:

InvalidPathException - if the path is invalid

FileDoesNotExistException - if the path does not exist

UnavailableException

getNewBlockIdForFile

long getNewBlockIdForFile(AlluxioURI path)
                   throws FileDoesNotExistException,
                          InvalidPathException,
                          AccessControlException

Gets a new block id for the next block of a given file to write to.

This operation requires users to have WRITE permission on the path as this API is called when creating a new block for a file.

Parameters:

path - the path of the file to get the next block id for

Returns:

the next block id for the given file

Throws:

FileDoesNotExistException - if the file does not exist

InvalidPathException - if the given path is not valid

AccessControlException - if permission checking fails

getMountTable

Map<String,MountPointInfo> getMountTable()

Returns:

a copy of the current mount table

getMountPointInfo

@Deprecated
MountPointInfo getMountPointInfo(AlluxioURI path)
                                      throws InvalidPathException

Deprecated. will be removed after 2.0

Gets the mount point information of an Alluxio path.

Parameters:

path - an Alluxio path which must be a mount point

Returns:

the mount point information

Throws:

InvalidPathException

getDisplayMountPointInfo

MountPointInfo getDisplayMountPointInfo(AlluxioURI path)
                                 throws InvalidPathException

Gets the mount point information of an Alluxio path for display purpose.

Parameters:

path - an Alluxio path which must be a mount point

Returns:

the mount point information

Throws:

InvalidPathException

getNumberOfPaths

int getNumberOfPaths()

Returns:

the number of files and directories

getNumberOfPinnedFiles

int getNumberOfPinnedFiles()

Returns:

the number of pinned files and directories

delete

void delete(AlluxioURI path,
            DeleteOptions options)
     throws IOException,
            FileDoesNotExistException,
            DirectoryNotEmptyException,
            InvalidPathException,
            AccessControlException

Deletes a given path.

This operation requires user to have WRITE permission on the parent of the path.

Parameters:

path - the path to delete

options - method options

Throws:

DirectoryNotEmptyException - if recursive is false and the file is a nonempty directory

FileDoesNotExistException - if the file does not exist

AccessControlException - if permission checking fails

InvalidPathException - if the path is invalid

IOException

getFileBlockInfoList

List<FileBlockInfo> getFileBlockInfoList(AlluxioURI path)
                                  throws FileDoesNotExistException,
                                         InvalidPathException,
                                         AccessControlException,
                                         UnavailableException

Gets the FileBlockInfo for all blocks of a file. If path is a directory, an exception is thrown.

This operation requires the client user to have READ permission on the the path.

Parameters:

path - the path to get the info for

Returns:

a list of FileBlockInfo for all the blocks of the given path

Throws:

FileDoesNotExistException - if the file does not exist or path is a directory

InvalidPathException - if the path of the given file is invalid

AccessControlException - if permission checking fails

UnavailableException

getInAlluxioFiles

List<AlluxioURI> getInAlluxioFiles()
                            throws UnavailableException

Returns:

absolute paths of all in-Alluxio files

Throws:

UnavailableException

getInMemoryFiles

List<AlluxioURI> getInMemoryFiles()
                           throws UnavailableException

Returns:

absolute paths of all in-memory files

Throws:

UnavailableException

createDirectory

long createDirectory(AlluxioURI path,
                     CreateDirectoryOptions options)
              throws InvalidPathException,
                     FileAlreadyExistsException,
                     IOException,
                     AccessControlException,
                     FileDoesNotExistException

Creates a directory for a given path.

This operation requires the client user to have WRITE permission on the parent of the path.

Parameters:

path - the path of the directory

options - method options

Returns:

the id of the created directory

Throws:

InvalidPathException - when the path is invalid

FileAlreadyExistsException - when there is already a file at path

AccessControlException - if permission checking fails

FileDoesNotExistException - if the parent of the path does not exist and the recursive option is false

IOException

rename

void rename(AlluxioURI srcPath,
            AlluxioURI dstPath,
            RenameOptions options)
     throws FileAlreadyExistsException,
            FileDoesNotExistException,
            InvalidPathException,
            IOException,
            AccessControlException

Renames a file to a destination.

This operation requires users to have WRITE permission on the parent of the src path, and WRITE permission on the parent of the dst path.

Parameters:

srcPath - the source path to rename

dstPath - the destination path to rename the file to

options - method options

Throws:

FileDoesNotExistException - if a non-existent file is encountered

InvalidPathException - if an invalid path is encountered

AccessControlException - if permission checking fails

FileAlreadyExistsException - if the file already exists

IOException

free

void free(AlluxioURI path,
          FreeOptions options)
   throws FileDoesNotExistException,
          InvalidPathException,
          AccessControlException,
          UnexpectedAlluxioException,
          IOException

Frees or evicts all of the blocks of the file from alluxio storage. If the given file is a directory, and the 'recursive' flag is enabled, all descendant files will also be freed.

This operation requires users to have READ permission on the path.

Parameters:

path - the path to free

options - options to free

Throws:

FileDoesNotExistException - if the file does not exist

AccessControlException - if permission checking fails

InvalidPathException - if the given path is invalid

UnexpectedAlluxioException - if the file or directory can not be freed

IOException

getPath

AlluxioURI getPath(long fileId)
            throws FileDoesNotExistException

Gets the path of a file with the given id.

Parameters:

fileId - the id of the file to look up

Returns:

the path of the file

Throws:

FileDoesNotExistException - raise if the file does not exist

getPinIdList

Set<Long> getPinIdList()

Returns:

the set of inode ids which are pinned

getUfsAddress

String getUfsAddress()

Returns:

the ufs address for this master

getUfsInfo

UfsInfo getUfsInfo(long mountId)

Parameters:

mountId - the mount id to query

Returns:

the ufs information for the given mount id

getWhiteList

List<String> getWhiteList()

Returns:

the white list

getLostFiles

List<Long> getLostFiles()

Returns:

all the files lost on the workers

reportLostFile

void reportLostFile(long fileId)
             throws FileDoesNotExistException,
                    UnavailableException

Reports a file as lost.

Parameters:

fileId - the id of the file

Throws:

FileDoesNotExistException - if the file does not exist

UnavailableException

loadMetadata

long loadMetadata(AlluxioURI path,
                  LoadMetadataOptions options)
           throws BlockInfoException,
                  FileDoesNotExistException,
                  InvalidPathException,
                  InvalidFileSizeException,
                  FileAlreadyCompletedException,
                  IOException,
                  AccessControlException

Loads metadata for the object identified by the given path from UFS into Alluxio.

This operation requires users to have WRITE permission on the path and its parent path if path is a file, or WRITE permission on the parent path if path is a directory.

Parameters:

path - the path for which metadata should be loaded

options - the load metadata options

Returns:

the file id of the loaded path

Throws:

BlockInfoException - if an invalid block size is encountered

FileDoesNotExistException - if there is no UFS path

InvalidPathException - if invalid path is encountered

InvalidFileSizeException - if invalid file size is encountered

FileAlreadyCompletedException - if the file is already completed

AccessControlException - if permission checking fails

IOException

mount

void mount(AlluxioURI alluxioPath,
           AlluxioURI ufsPath,
           MountOptions options)
    throws FileAlreadyExistsException,
           FileDoesNotExistException,
           InvalidPathException,
           IOException,
           AccessControlException

Mounts a UFS path onto an Alluxio path.

This operation requires users to have WRITE permission on the parent of the Alluxio path.

Parameters:

alluxioPath - the Alluxio path to mount to

ufsPath - the UFS path to mount

options - the mount options

Throws:

FileAlreadyExistsException - if the path to be mounted to already exists

FileDoesNotExistException - if the parent of the path to be mounted to does not exist

InvalidPathException - if an invalid path is encountered

AccessControlException - if the permission check fails

IOException

unmount

void unmount(AlluxioURI alluxioPath)
      throws FileDoesNotExistException,
             InvalidPathException,
             IOException,
             AccessControlException

Unmounts a UFS path previously mounted onto an Alluxio path.

This operation requires users to have WRITE permission on the parent of the Alluxio path.

Parameters:

alluxioPath - the Alluxio path to unmount, must be a mount point

Throws:

FileDoesNotExistException - if the path to be mounted does not exist

InvalidPathException - if the given path is not a mount point

AccessControlException - if the permission check fails

IOException

resetFile

void resetFile(long fileId)
        throws UnexpectedAlluxioException,
               FileDoesNotExistException,
               InvalidPathException,
               AccessControlException,
               IOException

Resets a file. It first free the whole file, and then reinitializes it.

Parameters:

fileId - the id of the file

Throws:

FileDoesNotExistException - if the file does not exist

AccessControlException - if permission checking fails

InvalidPathException - if the path is invalid for the id of the file

UnexpectedAlluxioException - if the file or directory can not be freed

IOException

setAttribute

void setAttribute(AlluxioURI path,
                  SetAttributeOptions options)
           throws FileDoesNotExistException,
                  AccessControlException,
                  InvalidPathException,
                  IOException

Sets the file attribute.

This operation requires users to have WRITE permission on the path. In addition, the client user must be a super user when setting the owner, and must be a super user or the owner when setting the group or permission.

Parameters:

path - the path to set attribute for

options - attributes to be set, see SetAttributeOptions

Throws:

FileDoesNotExistException - if the file does not exist

AccessControlException - if permission checking fails

InvalidPathException - if the given path is invalid

IOException

scheduleAsyncPersistence

void scheduleAsyncPersistence(AlluxioURI path)
                       throws AlluxioException,
                              UnavailableException

Schedules a file for async persistence.

Parameters:

path - the path of the file for persistence

Throws:

AlluxioException

UnavailableException

updateUfsMode

void updateUfsMode(AlluxioURI ufsPath,
                   UnderFileSystem.UfsMode ufsMode)
            throws InvalidPathException,
                   InvalidArgumentException,
                   UnavailableException,
                   AccessControlException

Update the operation mode for the given ufs path under one or more mount points.

Parameters:

ufsPath - the physical ufs path

ufsMode - the ufs operation mode

Throws:

InvalidPathException - if ufs path is not used by any mount point

InvalidArgumentException - if arguments for the method are invalid

UnavailableException

AccessControlException

validateInodeBlocks

void validateInodeBlocks(boolean repair)
                  throws UnavailableException

Checks the integrity of the inodes with respect to the blocks of the system.

Parameters:

repair - if true, will attempt to repair the state of the system when inconsistencies are discovered

Throws:

UnavailableException - if the repair attempt fails

workerHeartbeat

FileSystemCommand workerHeartbeat(long workerId,
                                  List<Long> persistedFiles,
                                  WorkerHeartbeatOptions options)
                           throws FileDoesNotExistException,
                                  InvalidPathException,
                                  AccessControlException,
                                  IOException

Instructs a worker to persist the files.

Needs WRITE permission on the list of files.

Parameters:

workerId - the id of the worker that heartbeats

persistedFiles - the files that persisted on the worker

options - the method options

Returns:

the command for persisting the blocks of a file

Throws:

FileDoesNotExistException - if the file does not exist

InvalidPathException - if the file path corresponding to the file id is invalid

AccessControlException - if permission checking fails

IOException

getWorkerInfoList

List<WorkerInfo> getWorkerInfoList()
                            throws UnavailableException

Returns:

a list of WorkerInfo objects representing the workers in Alluxio

Throws:

UnavailableException