alluxio.client.file

Class MetadataCachingFileSystem

java.lang.Object

alluxio.client.file.DelegatingFileSystem

alluxio.client.file.MetadataCachingFileSystem

All Implemented Interfaces:

FileSystem, Closeable, AutoCloseable

@ThreadSafe
public class MetadataCachingFileSystem
extends DelegatingFileSystem

FileSystem implementation with the capability of caching metadata of paths.

Nested Class Summary

Nested classes/interfaces inherited from interface alluxio.client.file.FileSystem

FileSystem.Factory

Field Summary

Fields inherited from class alluxio.client.file.DelegatingFileSystem

mDelegatedFileSystem

Constructor Summary

Constructors

Constructor and Description
MetadataCachingFileSystem(FileSystem fileSystem, FileSystemContext context)

Method Summary

Modifier and Type	Method and Description
void	asyncUpdateFileAccessTime(AlluxioURI path) Asynchronously update file's last access time.
void	close()
void	createDirectory(AlluxioURI path, CreateDirectoryPOptions options) Creates a directory.
FileOutStream	createFile(AlluxioURI path, CreateFilePOptions options) Creates a file.
void	delete(AlluxioURI path, DeletePOptions options) Deletes a file or a directory.
void	dropMetadataCache(AlluxioURI uri) Best efforts to drops metadata cache of a given uri, all its ancestors and descendants.
void	dropMetadataCacheAll() Drop all metadata cache.
List<BlockLocationInfo>	getBlockLocations(AlluxioURI path) Builds a list of BlockLocationInfo for the given file.
long	getMetadataCacheSize()
URIStatus	getStatus(AlluxioURI path, GetStatusPOptions options) Gets the URIStatus object that represents the metadata of an Alluxio path.
void	iterateStatus(AlluxioURI path, ListStatusPOptions options, java.util.function.Consumer<? super URIStatus> action) Performs a specific action on each URIStatus in the result of FileSystem.listStatus(alluxio.AlluxioURI).
List<URIStatus>	listStatus(AlluxioURI path, ListStatusPOptions options) If the path is a directory, returns the URIStatus of all the direct entries in it.
FileInStream	openFile(AlluxioURI path, OpenFilePOptions options) Opens a file for reading.
void	rename(AlluxioURI src, AlluxioURI dst, RenamePOptions options) Renames an existing Alluxio path to another Alluxio path in Alluxio.

Methods inherited from class alluxio.client.file.DelegatingFileSystem

checkAccess, exists, free, getBlockLocations, getConf, getJobProgress, getMountTable, getSyncPathList, getUnderlyingFileSystem, isClosed, listStatusPartial, loadMetadata, mount, needsSync, openFile, persist, reverseResolve, setAcl, setAttribute, startSync, stopJob, stopSync, submitJob, unmount, updateMount

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface alluxio.client.file.FileSystem

createDirectory, createFile, delete, exists, free, getMountTable, getStatus, iterateStatus, listStatus, loadMetadata, mount, openFile, persist, rename, setAcl, setAttribute, unmount

Constructor Detail

MetadataCachingFileSystem

public MetadataCachingFileSystem(FileSystem fileSystem,
                                 FileSystemContext context)

Parameters:

fileSystem - the file system

context - the fs context

Method Detail

createDirectory

public void createDirectory(AlluxioURI path,
                            CreateDirectoryPOptions options)
                     throws FileAlreadyExistsException,
                            InvalidPathException,
                            IOException,
                            AlluxioException

Description copied from interface: FileSystem

Creates a directory.

Specified by:

createDirectory in interface FileSystem

Overrides:

createDirectory in class DelegatingFileSystem

Parameters:

path - the path of the directory to create in Alluxio space

options - options to associate with this operation

Throws:

FileAlreadyExistsException - if there is already a file or directory at the given path

InvalidPathException - if the path is invalid

IOException

AlluxioException

createFile

public FileOutStream createFile(AlluxioURI path,
                                CreateFilePOptions options)
                         throws IOException,
                                AlluxioException

Description copied from interface: FileSystem

Creates a file.

Specified by:

createFile in interface FileSystem

Overrides:

createFile in class DelegatingFileSystem

Parameters:

path - the path of the file to create in Alluxio space

options - options to associate with this operation

Returns:

a FileOutStream which will write data to the newly created file

Throws:

FileAlreadyExistsException - if there is already a file at the given path

InvalidPathException - if the path is invalid

IOException

AlluxioException

delete

public void delete(AlluxioURI path,
                   DeletePOptions options)
            throws IOException,
                   AlluxioException

Description copied from interface: FileSystem

Deletes a file or a directory.

Specified by:

delete in interface FileSystem

Overrides:

delete in class DelegatingFileSystem

Parameters:

path - the path to delete in Alluxio space

options - options to associate with this operation

Throws:

FileDoesNotExistException - if the given path does not exist

DirectoryNotEmptyException - if recursive is false and the path is a nonempty directory

IOException

AlluxioException

rename

public void rename(AlluxioURI src,
                   AlluxioURI dst,
                   RenamePOptions options)
            throws IOException,
                   AlluxioException

Description copied from interface: FileSystem

Renames an existing Alluxio path to another Alluxio path in Alluxio. This operation will be propagated in the underlying storage if the path is persisted.

Specified by:

rename in interface FileSystem

Overrides:

rename in class DelegatingFileSystem

Parameters:

src - the path of the source, this must already exist

dst - the path of the destination, this path should not exist

options - options to associate with this operation

Throws:

FileDoesNotExistException - if the given file does not exist

IOException

AlluxioException

getBlockLocations

public List<BlockLocationInfo> getBlockLocations(AlluxioURI path)
                                          throws IOException,
                                                 AlluxioException

Description copied from interface: FileSystem

Builds a list of BlockLocationInfo for the given file. Each list item contains a list of WorkerNetAddress which allows a user to determine the physical location of a block of the given file stored within Alluxio. In the case where data is stored in a UFS, but not in Alluxio this function will only include a WorkerNetAddress if the block stored in the UFS is co-located with an Alluxio worker. However if there are no co-located Alluxio workers for the block, then the behavior is controlled by the PropertyKey.USER_UFS_BLOCK_LOCATION_ALL_FALLBACK_ENABLED . If this property is set to true then every Alluxio worker will be returned. Blocks which are stored in the UFS and are *not* co-located with any Alluxio worker will return an empty list. If the file block is within Alluxio *and* the UFS then this will only return Alluxio workers which currently store the block.

Specified by:

getBlockLocations in interface FileSystem

Overrides:

getBlockLocations in class DelegatingFileSystem

Parameters:

path - the path to get block info for

Returns:

a list of blocks with the workers whose hosts have the blocks. The blocks may not necessarily be stored in Alluxio. The blocks are returned in the order of their sequences in file.

Throws:

FileDoesNotExistException - if the given path does not exist

IOException

AlluxioException

getStatus

public URIStatus getStatus(AlluxioURI path,
                           GetStatusPOptions options)
                    throws FileDoesNotExistException,
                           IOException,
                           AlluxioException

Description copied from interface: FileSystem

Gets the URIStatus object that represents the metadata of an Alluxio path.

Specified by:

getStatus in interface FileSystem

Overrides:

getStatus in class DelegatingFileSystem

Parameters:

path - the path to obtain information about

options - options to associate with this operation

Returns:

the URIStatus of the file

Throws:

FileDoesNotExistException - if the path does not exist

IOException

AlluxioException

iterateStatus

public void iterateStatus(AlluxioURI path,
                          ListStatusPOptions options,
                          java.util.function.Consumer<? super URIStatus> action)
                   throws FileDoesNotExistException,
                          IOException,
                          AlluxioException

Description copied from interface: FileSystem

Performs a specific action on each URIStatus in the result of FileSystem.listStatus(alluxio.AlluxioURI). This method is preferred when iterating over directories with a large number of files or sub-directories inside. The caller can proceed with partial result without waiting for all result returned.

Specified by:

iterateStatus in interface FileSystem

Overrides:

iterateStatus in class DelegatingFileSystem

Parameters:

path - the path to list information about

options - options to associate with this operation

action - action to apply on each URIStatus

Throws:

FileDoesNotExistException - if the given path does not exist

IOException

AlluxioException

listStatus

public List<URIStatus> listStatus(AlluxioURI path,
                                  ListStatusPOptions options)
                           throws FileDoesNotExistException,
                                  IOException,
                                  AlluxioException

Description copied from interface: FileSystem

If the path is a directory, returns the URIStatus of all the direct entries in it. Otherwise returns a list with a single URIStatus element for the file.

Specified by:

listStatus in interface FileSystem

Overrides:

listStatus in class DelegatingFileSystem

Parameters:

path - the path to list information about

options - options to associate with this operation

Returns:

a list of URIStatuss containing information about the files and directories which are children of the given path

Throws:

FileDoesNotExistException - if the given path does not exist

IOException

AlluxioException

openFile

public FileInStream openFile(AlluxioURI path,
                             OpenFilePOptions options)
                      throws FileDoesNotExistException,
                             OpenDirectoryException,
                             FileIncompleteException,
                             IOException,
                             AlluxioException

Description copied from interface: FileSystem

Opens a file for reading.

Specified by:

openFile in interface FileSystem

Overrides:

openFile in class DelegatingFileSystem

Parameters:

path - the file to read from

options - options to associate with this operation

Returns:

a FileInStream for the given path

Throws:

FileDoesNotExistException - when path does not exist

OpenDirectoryException - when path is a directory

FileIncompleteException - when path is a file and is not completed yet

IOException

AlluxioException

asyncUpdateFileAccessTime

public void asyncUpdateFileAccessTime(AlluxioURI path)

Asynchronously update file's last access time.

Parameters:

path - the path to the file

close

public void close()
           throws IOException

Specified by:

close in interface Closeable

Specified by:

close in interface AutoCloseable

Overrides:

close in class DelegatingFileSystem

Throws:

IOException

dropMetadataCache

public void dropMetadataCache(AlluxioURI uri)

Best efforts to drops metadata cache of a given uri, all its ancestors and descendants.

Parameters:

uri - the uri need to drop metadata cache

dropMetadataCacheAll

public void dropMetadataCacheAll()

Drop all metadata cache.

getMetadataCacheSize

public long getMetadataCacheSize()

Returns:

metadata cache size