alluxio.client.file

Class DoraCacheFileSystem

java.lang.Object

alluxio.client.file.DelegatingFileSystem

alluxio.client.file.DoraCacheFileSystem

All Implemented Interfaces:

FileSystem, Closeable, AutoCloseable

public class DoraCacheFileSystem
extends DelegatingFileSystem

Dora Cache file system implementation.

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static class	DoraCacheFileSystem.DoraCacheFileSystemFactory DoraCacheFileSystem Factory.

Nested classes/interfaces inherited from interface alluxio.client.file.FileSystem

FileSystem.Factory

Field Summary

Fields

Modifier and Type	Field and Description
static int	DUMMY_MOUNT_ID
protected FileSystemContext	mFsContext
static DoraCacheFileSystem.DoraCacheFileSystemFactory	sDoraCacheFileSystemFactory

Fields inherited from class alluxio.client.file.DelegatingFileSystem

mDelegatedFileSystem

Constructor Summary

Constructors

Modifier	Constructor and Description
	DoraCacheFileSystem(FileSystem fs, FileSystemContext context) Wraps a file system instance to forward messages.
protected	DoraCacheFileSystem(FileSystem fs, FileSystemContext context, DoraCacheClient doraCacheClient)

Method Summary

Modifier and Type	Method and Description
Map<String,List<WorkerNetAddress>>	checkFileLocation(AlluxioURI path) Check the location of the specified path.
Map<String,List<WorkerNetAddress>>	checkFileLocation(AlluxioURI path, GetStatusPOptions options) Check the location of the specified path.
AlluxioURI	convertToAlluxioPath(AlluxioURI ufsPath) Converts the UFS path back to Alluxio path.
AlluxioURI	convertToUfsPath(AlluxioURI alluxioPath) Converts the Alluxio based path to UfsBaseFileSystem based path if needed.
void	createDirectory(AlluxioURI path, CreateDirectoryPOptions options) Creates a directory.
FileOutStream	createFile(AlluxioURI alluxioPath, CreateFilePOptions options) Creates a file.
void	delete(AlluxioURI path, DeletePOptions options) Deletes a file or a directory.
boolean	exists(AlluxioURI path, ExistsPOptions options) Checks whether a path exists in Alluxio space.
List<BlockLocationInfo>	getBlockLocations(AlluxioURI path) Get the location information of the specified file.
List<BlockLocationInfo>	getBlockLocations(URIStatus status) Get the location information of the specified file.
DoraCacheFileSystem	getDoraCacheFileSystem()
URIStatus	getStatus(AlluxioURI path, GetStatusPOptions options) Gets the URIStatus object that represents the metadata of an Alluxio path.
WorkerNetAddress	getWorkerNetAddress(AlluxioURI path) Get the worker address which the specified file locates at.
void	iterateStatus(AlluxioURI path, ListStatusPOptions options, java.util.function.Consumer<? super URIStatus> action) Performs a specific action on each URIStatus in the result of FileSystem.listStatus(alluxio.AlluxioURI).
List<URIStatus>	listStatus(AlluxioURI path, ListStatusPOptions options) If the path is a directory, returns the URIStatus of all the direct entries in it.
FileInStream	openFile(AlluxioURI path, OpenFilePOptions options) Opens a file for reading.
FileInStream	openFile(URIStatus status, OpenFilePOptions options) Opens a file for reading.
PositionReader	openPositionRead(AlluxioURI path, OpenFilePOptions options) Opens a file for position reading.
PositionReader	openPositionRead(URIStatus status, OpenFilePOptions options) Opens a file for reading.
void	rename(AlluxioURI src, AlluxioURI dst, RenamePOptions options) Renames an existing Alluxio path to another Alluxio path in Alluxio.
void	setAttribute(AlluxioURI path, SetAttributePOptions options) Sets any number of a path's attributes, such as TTL and pin status.
void	setUfsFallbackEnabled(boolean enabled) Dora Cache file system implementation.

Methods inherited from class alluxio.client.file.DelegatingFileSystem

checkAccess, close, free, getConf, getJobProgress, getMountTable, getUfsBaseFileSystem, getUnderlyingFileSystem, isClosed, listStatusPartial, loadMetadata, mount, needsSync, persist, reverseResolve, setAcl, stopJob, submitJob, unmount, updateMount

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface alluxio.client.file.FileSystem

createDirectory, createFile, delete, exists, free, getMountTable, getStatus, iterateStatus, listStatus, loadMetadata, mount, openFile, openPositionRead, persist, rename, setAcl, setAttribute, unmount

Field Detail

DUMMY_MOUNT_ID

public static final int DUMMY_MOUNT_ID

See Also:

Constant Field Values

sDoraCacheFileSystemFactory

public static DoraCacheFileSystem.DoraCacheFileSystemFactory sDoraCacheFileSystemFactory

mFsContext

protected final FileSystemContext mFsContext

Constructor Detail

DoraCacheFileSystem

public DoraCacheFileSystem(FileSystem fs,
                           FileSystemContext context)

Wraps a file system instance to forward messages.

Parameters:

fs - the underlying file system

context -

DoraCacheFileSystem

protected DoraCacheFileSystem(FileSystem fs,
                              FileSystemContext context,
                              DoraCacheClient doraCacheClient)

Method Detail

getStatus

public URIStatus getStatus(AlluxioURI path,
                           GetStatusPOptions options)
                    throws IOException,
                           AlluxioException

Description copied from interface: FileSystem

Gets the URIStatus object that represents the metadata of an Alluxio path.

Specified by:

getStatus in interface FileSystem

Overrides:

getStatus in class DelegatingFileSystem

Parameters:

path - the path to obtain information about

options - options to associate with this operation

Returns:

the URIStatus of the file

Throws:

FileDoesNotExistException - if the path does not exist

IOException

AlluxioException

openFile

public FileInStream openFile(AlluxioURI path,
                             OpenFilePOptions options)
                      throws IOException,
                             AlluxioException

Description copied from interface: FileSystem

Opens a file for reading.

Specified by:

openFile in interface FileSystem

Overrides:

openFile in class DelegatingFileSystem

Parameters:

path - the file to read from

options - options to associate with this operation

Returns:

a FileInStream for the given path

Throws:

FileDoesNotExistException - when path does not exist

OpenDirectoryException - when path is a directory

FileIncompleteException - when path is a file and is not completed yet

IOException

AlluxioException

openFile

public FileInStream openFile(URIStatus status,
                             OpenFilePOptions options)
                      throws IOException,
                             AlluxioException

Description copied from interface: FileSystem

Opens a file for reading.

Specified by:

openFile in interface FileSystem

Overrides:

openFile in class DelegatingFileSystem

Parameters:

status - status of the file to read from

options - options to associate with this operation

Returns:

a FileInStream for the given path

Throws:

FileDoesNotExistException - when path does not exist

OpenDirectoryException - when path is a directory

FileIncompleteException - when path is a file and is not completed yet

IOException

AlluxioException

openPositionRead

public PositionReader openPositionRead(AlluxioURI path,
                                       OpenFilePOptions options)

Description copied from interface: FileSystem

Opens a file for position reading.

Specified by:

openPositionRead in interface FileSystem

Overrides:

openPositionRead in class DelegatingFileSystem

Parameters:

path - the file to read from

options - options to associate with this operation

Returns:

a PositionReader for the given path

openPositionRead

public PositionReader openPositionRead(URIStatus status,
                                       OpenFilePOptions options)

Description copied from interface: FileSystem

Opens a file for reading.

Specified by:

openPositionRead in interface FileSystem

Overrides:

openPositionRead in class DelegatingFileSystem

Parameters:

status - status of the file to read from

options - options to associate with this operation

Returns:

a PositionReader for the given path

listStatus

public List<URIStatus> listStatus(AlluxioURI path,
                                  ListStatusPOptions options)
                           throws FileDoesNotExistException,
                                  IOException,
                                  AlluxioException

Description copied from interface: FileSystem

If the path is a directory, returns the URIStatus of all the direct entries in it. Otherwise returns a list with a single URIStatus element for the file.

Specified by:

listStatus in interface FileSystem

Overrides:

listStatus in class DelegatingFileSystem

Parameters:

path - the path to list information about

options - options to associate with this operation

Returns:

a list of URIStatuss containing information about the files and directories which are children of the given path

Throws:

FileDoesNotExistException - if the given path does not exist

IOException

AlluxioException

createFile

public FileOutStream createFile(AlluxioURI alluxioPath,
                                CreateFilePOptions options)
                         throws FileAlreadyExistsException,
                                InvalidPathException,
                                IOException,
                                AlluxioException

Description copied from interface: FileSystem

Creates a file.

Specified by:

createFile in interface FileSystem

Overrides:

createFile in class DelegatingFileSystem

Parameters:

alluxioPath - the path of the file to create in Alluxio space

options - options to associate with this operation

Returns:

a FileOutStream which will write data to the newly created file

Throws:

FileAlreadyExistsException - if there is already a file at the given path

InvalidPathException - if the path is invalid

IOException

AlluxioException

createDirectory

public void createDirectory(AlluxioURI path,
                            CreateDirectoryPOptions options)
                     throws FileAlreadyExistsException,
                            InvalidPathException,
                            IOException,
                            AlluxioException

Description copied from interface: FileSystem

Creates a directory.

Specified by:

createDirectory in interface FileSystem

Overrides:

createDirectory in class DelegatingFileSystem

Parameters:

path - the path of the directory to create in Alluxio space

options - options to associate with this operation

Throws:

FileAlreadyExistsException - if there is already a file or directory at the given path

InvalidPathException - if the path is invalid

IOException

AlluxioException

delete

public void delete(AlluxioURI path,
                   DeletePOptions options)
            throws DirectoryNotEmptyException,
                   FileDoesNotExistException,
                   IOException,
                   AlluxioException

Description copied from interface: FileSystem

Deletes a file or a directory.

Specified by:

delete in interface FileSystem

Overrides:

delete in class DelegatingFileSystem

Parameters:

path - the path to delete in Alluxio space

options - options to associate with this operation

Throws:

DirectoryNotEmptyException - if recursive is false and the path is a nonempty directory

FileDoesNotExistException - if the given path does not exist

IOException

AlluxioException

rename

public void rename(AlluxioURI src,
                   AlluxioURI dst,
                   RenamePOptions options)
            throws FileDoesNotExistException,
                   IOException,
                   AlluxioException

Description copied from interface: FileSystem

Renames an existing Alluxio path to another Alluxio path in Alluxio. This operation will be propagated in the underlying storage if the path is persisted.

Specified by:

rename in interface FileSystem

Overrides:

rename in class DelegatingFileSystem

Parameters:

src - the path of the source, this must already exist

dst - the path of the destination, this path should not exist

options - options to associate with this operation

Throws:

FileDoesNotExistException - if the given file does not exist

IOException

AlluxioException

iterateStatus

public void iterateStatus(AlluxioURI path,
                          ListStatusPOptions options,
                          java.util.function.Consumer<? super URIStatus> action)
                   throws FileDoesNotExistException,
                          IOException,
                          AlluxioException

Description copied from interface: FileSystem

Performs a specific action on each URIStatus in the result of FileSystem.listStatus(alluxio.AlluxioURI). This method is preferred when iterating over directories with a large number of files or sub-directories inside. The caller can proceed with partial result without waiting for all result returned.

Specified by:

iterateStatus in interface FileSystem

Overrides:

iterateStatus in class DelegatingFileSystem

Parameters:

path - the path to list information about

options - options to associate with this operation

action - action to apply on each URIStatus

Throws:

FileDoesNotExistException - if the given path does not exist

IOException

AlluxioException

exists

public boolean exists(AlluxioURI path,
                      ExistsPOptions options)
               throws InvalidPathException,
                      IOException,
                      AlluxioException

Description copied from interface: FileSystem

Checks whether a path exists in Alluxio space.

Specified by:

exists in interface FileSystem

Overrides:

exists in class DelegatingFileSystem

Parameters:

path - the path in question

options - options to associate with this operation

Returns:

true if the path exists, false otherwise

Throws:

InvalidPathException - if the path is invalid

IOException

AlluxioException

setAttribute

public void setAttribute(AlluxioURI path,
                         SetAttributePOptions options)
                  throws FileDoesNotExistException,
                         IOException,
                         AlluxioException

Description copied from interface: FileSystem

Sets any number of a path's attributes, such as TTL and pin status.

Specified by:

setAttribute in interface FileSystem

Overrides:

setAttribute in class DelegatingFileSystem

Parameters:

path - the path to set attributes for

options - options to associate with this operation

Throws:

FileDoesNotExistException - if the given file does not exist

IOException

AlluxioException

convertToUfsPath

public AlluxioURI convertToUfsPath(AlluxioURI alluxioPath)

Converts the Alluxio based path to UfsBaseFileSystem based path if needed.

Parameters:

alluxioPath - Alluxio based path

Returns:

UfsBaseFileSystem based full path

convertToAlluxioPath

public AlluxioURI convertToAlluxioPath(AlluxioURI ufsPath)
                                throws InvalidPathException

Converts the UFS path back to Alluxio path.

This is the opposite operation to convertToUfsPath(AlluxioURI).

Parameters:

ufsPath - UfsBaseFileSystem based full path

Returns:

an Alluxio path

Throws:

InvalidPathException - if ufsPath is not a child of the UFS mounted at Alluxio root

getWorkerNetAddress

public WorkerNetAddress getWorkerNetAddress(AlluxioURI path)

Get the worker address which the specified file locates at.

Parameters:

path - the file path

Returns:

the worker address which the file locates at

checkFileLocation

public Map<String,List<WorkerNetAddress>> checkFileLocation(AlluxioURI path)
                                                     throws IOException

Check the location of the specified path.

Parameters:

path - the file path

Returns:

a map that maps the file path to a list of workers

Throws:

IOException

checkFileLocation

public Map<String,List<WorkerNetAddress>> checkFileLocation(AlluxioURI path,
                                                            GetStatusPOptions options)
                                                     throws IOException

Check the location of the specified path.

Parameters:

path - the file path

options - the get status options

Returns:

a map that maps the file path to a list of workers

Throws:

IOException

getBlockLocations

public List<BlockLocationInfo> getBlockLocations(AlluxioURI path)
                                          throws IOException,
                                                 AlluxioException

Get the location information of the specified file.

Specified by:

getBlockLocations in interface FileSystem

Overrides:

getBlockLocations in class DelegatingFileSystem

Parameters:

path - the path to get the location information

Returns:

the location information of the specified file

Throws:

IOException

AlluxioException

FileDoesNotExistException - if the given path does not exist

getBlockLocations

public List<BlockLocationInfo> getBlockLocations(URIStatus status)
                                          throws IOException,
                                                 AlluxioException

Get the location information of the specified file.

Specified by:

getBlockLocations in interface FileSystem

Overrides:

getBlockLocations in class DelegatingFileSystem

Parameters:

status - the uri of the file

Returns:

the location information of the specified file

Throws:

IOException

AlluxioException

setUfsFallbackEnabled

public void setUfsFallbackEnabled(boolean enabled)

Dora Cache file system implementation.

Parameters:

enabled - is ufs fall back enabled

getDoraCacheFileSystem

@Nullable
public DoraCacheFileSystem getDoraCacheFileSystem()

Specified by:

getDoraCacheFileSystem in interface FileSystem

Overrides:

getDoraCacheFileSystem in class DelegatingFileSystem

Returns:

the instance of DoraCacheFileSystem