alluxio.hadoop

Class AlluxioHdfsFileSystem

java.lang.Object

alluxio.hadoop.AlluxioHdfsFileSystem

All Implemented Interfaces:

FileSystem, Closeable, AutoCloseable

public class AlluxioHdfsFileSystem
extends Object
implements FileSystem

A wrapper class to translate Hadoop FileSystem to Alluxio FileSystem. This class is only to be used internally and most methods are not implemented.

Nested Class Summary

Nested classes/interfaces inherited from interface alluxio.client.file.FileSystem

FileSystem.Factory

Constructor Summary

Constructors

Constructor and Description
AlluxioHdfsFileSystem(org.apache.hadoop.fs.FileSystem fileSystem, org.apache.hadoop.conf.Configuration conf)

Method Summary

Modifier and Type	Method and Description
void	close()
void	createDirectory(AlluxioURI alluxioURI, CreateDirectoryPOptions options) Creates a directory.
FileOutStream	createFile(AlluxioURI alluxioURI, CreateFilePOptions options) Creates a file.
void	delete(AlluxioURI alluxioURI, DeletePOptions options) Deletes a file or a directory.
boolean	exists(AlluxioURI alluxioURI, ExistsPOptions options) Checks whether a path exists in Alluxio space.
void	free(AlluxioURI alluxioURI, FreePOptions options) Evicts any data under the given path from Alluxio space, but does not delete the data from the UFS.
List<BlockLocationInfo>	getBlockLocations(AlluxioURI alluxioURI) Builds a list of BlockLocationInfo for the given file.
AlluxioConfiguration	getConf()
Map<String,MountPointInfo>	getMountTable() Lists all mount points and their corresponding under storage addresses.
URIStatus	getStatus(AlluxioURI alluxioURI, GetStatusPOptions options) Gets the URIStatus object that represents the metadata of an Alluxio path.
List<SyncPointInfo>	getSyncPathList() Lists all the actively synced paths.
boolean	isClosed() If there are operations currently running and close is called concurrently the behavior is undefined.
List<URIStatus>	listStatus(AlluxioURI alluxioURI, ListStatusPOptions options) If the path is a directory, returns the URIStatus of all the direct entries in it.
void	mount(AlluxioURI alluxioURI1, AlluxioURI alluxioURI2, MountPOptions options) Mounts a UFS subtree to the given Alluxio path.
FileInStream	openFile(AlluxioURI alluxioURI, OpenFilePOptions options) Opens a file for reading.
FileInStream	openFile(URIStatus uriStatus, OpenFilePOptions options) Opens a file for reading.
void	persist(AlluxioURI alluxioURI, ScheduleAsyncPersistencePOptions options) Schedules the given path to be asynchronously persisted to the under file system.
void	rename(AlluxioURI source, AlluxioURI destination, RenamePOptions options) Renames an existing Alluxio path to another Alluxio path in Alluxio.
AlluxioURI	reverseResolve(AlluxioURI alluxioURI) Reverse resolve a ufs uri.
void	setAcl(AlluxioURI alluxioURI, SetAclAction action, List<AclEntry> entries, SetAclPOptions options) Sets the ACL for a path.
void	setAttribute(AlluxioURI alluxioURI, SetAttributePOptions options) Sets any number of a path's attributes, such as TTL and pin status.
void	startSync(AlluxioURI alluxioURI) Starts the active syncing process on an Alluxio path.
void	stopSync(AlluxioURI alluxioURI) Stops the active syncing process on an Alluxio path.
void	unmount(AlluxioURI alluxioURI, UnmountPOptions options) Unmounts a UFS subtree identified by the given Alluxio path.
void	updateMount(AlluxioURI alluxioURI, MountPOptions options) Updates the options for an existing mount point.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface alluxio.client.file.FileSystem

createDirectory, createFile, delete, exists, free, getStatus, listStatus, mount, openFile, persist, rename, setAcl, setAttribute, unmount

Constructor Detail

AlluxioHdfsFileSystem

public AlluxioHdfsFileSystem(org.apache.hadoop.fs.FileSystem fileSystem,
                             org.apache.hadoop.conf.Configuration conf)

Parameters:

fileSystem - hadoop file system

conf - hadoop configuration

Method Detail

createDirectory

public void createDirectory(AlluxioURI alluxioURI,
                            CreateDirectoryPOptions options)
                     throws IOException

Description copied from interface: FileSystem

Creates a directory.

Specified by:

createDirectory in interface FileSystem

Parameters:

alluxioURI - the path of the directory to create in Alluxio space

options - options to associate with this operation

Throws:

IOException

openFile

public FileInStream openFile(AlluxioURI alluxioURI,
                             OpenFilePOptions options)
                      throws IOException

Description copied from interface: FileSystem

Opens a file for reading.

Specified by:

openFile in interface FileSystem

Parameters:

alluxioURI - the file to read from

options - options to associate with this operation

Returns:

a FileInStream for the given path

Throws:

IOException

openFile

public FileInStream openFile(URIStatus uriStatus,
                             OpenFilePOptions options)
                      throws IOException

Description copied from interface: FileSystem

Opens a file for reading.

Specified by:

openFile in interface FileSystem

Parameters:

uriStatus - status of the file to read from

options - options to associate with this operation

Returns:

a FileInStream for the given path

Throws:

IOException

createFile

public FileOutStream createFile(AlluxioURI alluxioURI,
                                CreateFilePOptions options)

Description copied from interface: FileSystem

Creates a file.

Specified by:

createFile in interface FileSystem

Parameters:

alluxioURI - the path of the file to create in Alluxio space

options - options to associate with this operation

Returns:

a FileOutStream which will write data to the newly created file

delete

public void delete(AlluxioURI alluxioURI,
                   DeletePOptions options)

Description copied from interface: FileSystem

Deletes a file or a directory.

Specified by:

delete in interface FileSystem

Parameters:

alluxioURI - the path to delete in Alluxio space

options - options to associate with this operation

exists

public boolean exists(AlluxioURI alluxioURI,
                      ExistsPOptions options)

Description copied from interface: FileSystem

Checks whether a path exists in Alluxio space.

Specified by:

exists in interface FileSystem

Parameters:

alluxioURI - the path in question

options - options to associate with this operation

Returns:

true if the path exists, false otherwise

getConf

public AlluxioConfiguration getConf()

Specified by:

getConf in interface FileSystem

Returns:

the configuration which the FileSystem is using to connect to Alluxio

getStatus

public URIStatus getStatus(AlluxioURI alluxioURI,
                           GetStatusPOptions options)
                    throws IOException

Description copied from interface: FileSystem

Gets the URIStatus object that represents the metadata of an Alluxio path.

Specified by:

getStatus in interface FileSystem

Parameters:

alluxioURI - the path to obtain information about

options - options to associate with this operation

Returns:

the URIStatus of the file

Throws:

IOException

listStatus

public List<URIStatus> listStatus(AlluxioURI alluxioURI,
                                  ListStatusPOptions options)
                           throws IOException

Description copied from interface: FileSystem

If the path is a directory, returns the URIStatus of all the direct entries in it. Otherwise returns a list with a single URIStatus element for the file.

Specified by:

listStatus in interface FileSystem

Parameters:

alluxioURI - the path to list information about

options - options to associate with this operation

Returns:

a list of URIStatuss containing information about the files and directories which are children of the given path

Throws:

IOException

isClosed

public boolean isClosed()

Description copied from interface: FileSystem

If there are operations currently running and close is called concurrently the behavior is undefined. After closing a FileSystem, any operations that are performed result in undefined behavior.

Specified by:

isClosed in interface FileSystem

Returns:

whether or not this FileSystem has been closed

free

public void free(AlluxioURI alluxioURI,
                 FreePOptions options)

Description copied from interface: FileSystem

Evicts any data under the given path from Alluxio space, but does not delete the data from the UFS. The metadata will still be present in Alluxio space after this operation.

Specified by:

free in interface FileSystem

Parameters:

alluxioURI - the path to free in Alluxio space

options - options to associate with this operation

getBlockLocations

public List<BlockLocationInfo> getBlockLocations(AlluxioURI alluxioURI)

Description copied from interface: FileSystem

Builds a list of BlockLocationInfo for the given file. Each list item contains a list of WorkerNetAddress which allows a user to determine the physical location of a block of the given file stored within Alluxio. In the case where data is stored in a UFS, but not in Alluxio this function will only include a WorkerNetAddress if the block stored in the UFS is co-located with an Alluxio worker. However if there are no co-located Alluxio workers for the block, then the behavior is controlled by the PropertyKey.USER_UFS_BLOCK_LOCATION_ALL_FALLBACK_ENABLED . If this property is set to true then every Alluxio worker will be returned. Blocks which are stored in the UFS and are *not* co-located with any Alluxio worker will return an empty list. If the file block is within Alluxio *and* the UFS then this will only return Alluxio workers which currently store the block.

Specified by:

getBlockLocations in interface FileSystem

Parameters:

alluxioURI - the path to get block info for

Returns:

a list of blocks with the workers whose hosts have the blocks. The blocks may not necessarily be stored in Alluxio. The blocks are returned in the order of their sequences in file.

mount

public void mount(AlluxioURI alluxioURI1,
                  AlluxioURI alluxioURI2,
                  MountPOptions options)

Description copied from interface: FileSystem

Mounts a UFS subtree to the given Alluxio path. The Alluxio path is expected not to exist as the method creates it. If the path already exists, a AlluxioException will be thrown. This method does not transfer any data or metadata from the UFS. It simply establishes the connection between the given Alluxio path and UFS path.

Specified by:

mount in interface FileSystem

Parameters:

alluxioURI1 - an Alluxio path to mount the data to

alluxioURI2 - a UFS path to mount the data from

options - options to associate with this operation

updateMount

public void updateMount(AlluxioURI alluxioURI,
                        MountPOptions options)

Description copied from interface: FileSystem

Updates the options for an existing mount point.

Specified by:

updateMount in interface FileSystem

Parameters:

alluxioURI - the Alluxio path of the mount point

options - options for this mount point

getMountTable

public Map<String,MountPointInfo> getMountTable()

Description copied from interface: FileSystem

Lists all mount points and their corresponding under storage addresses.

Specified by:

getMountTable in interface FileSystem

Returns:

a map from String to MountPointInfo

getSyncPathList

public List<SyncPointInfo> getSyncPathList()

Description copied from interface: FileSystem

Lists all the actively synced paths.

Specified by:

getSyncPathList in interface FileSystem

Returns:

a list of actively synced paths

persist

public void persist(AlluxioURI alluxioURI,
                    ScheduleAsyncPersistencePOptions options)

Description copied from interface: FileSystem

Schedules the given path to be asynchronously persisted to the under file system. To persist synchronously please see FileSystemUtils.persistAndWait(FileSystem, AlluxioURI, long).

Specified by:

persist in interface FileSystem

Parameters:

alluxioURI - the uri of the file to persist

options - the options to use when submitting persist the path

rename

public void rename(AlluxioURI source,
                   AlluxioURI destination,
                   RenamePOptions options)

Description copied from interface: FileSystem

Renames an existing Alluxio path to another Alluxio path in Alluxio. This operation will be propagated in the underlying storage if the path is persisted.

Specified by:

rename in interface FileSystem

Parameters:

source - the path of the source, this must already exist

destination - the path of the destination, this path should not exist

options - options to associate with this operation

reverseResolve

public AlluxioURI reverseResolve(AlluxioURI alluxioURI)

Description copied from interface: FileSystem

Reverse resolve a ufs uri.

Specified by:

reverseResolve in interface FileSystem

Parameters:

alluxioURI - the ufs uri

Returns:

the alluxio path for the ufsUri

setAcl

public void setAcl(AlluxioURI alluxioURI,
                   SetAclAction action,
                   List<AclEntry> entries,
                   SetAclPOptions options)

Description copied from interface: FileSystem

Sets the ACL for a path.

Specified by:

setAcl in interface FileSystem

Parameters:

alluxioURI - the path to set the ACL for

action - the set action to perform

entries - the ACL entries

options - options to associate with this operation

startSync

public void startSync(AlluxioURI alluxioURI)

Description copied from interface: FileSystem

Starts the active syncing process on an Alluxio path.

Specified by:

startSync in interface FileSystem

Parameters:

alluxioURI - the path to sync

stopSync

public void stopSync(AlluxioURI alluxioURI)

Description copied from interface: FileSystem

Stops the active syncing process on an Alluxio path.

Specified by:

stopSync in interface FileSystem

Parameters:

alluxioURI - the path to stop syncing

setAttribute

public void setAttribute(AlluxioURI alluxioURI,
                         SetAttributePOptions options)

Description copied from interface: FileSystem

Sets any number of a path's attributes, such as TTL and pin status.

Specified by:

setAttribute in interface FileSystem

Parameters:

alluxioURI - the path to set attributes for

options - options to associate with this operation

unmount

public void unmount(AlluxioURI alluxioURI,
                    UnmountPOptions options)

Description copied from interface: FileSystem

Unmounts a UFS subtree identified by the given Alluxio path. The Alluxio path match a previously mounted path. The contents of the subtree rooted at this path are removed from Alluxio but the corresponding UFS subtree is left untouched.

Specified by:

unmount in interface FileSystem

Parameters:

alluxioURI - an Alluxio path, this must be a mount point

options - options to associate with this operation

close

public void close()
           throws IOException

Specified by:

close in interface Closeable

Specified by:

close in interface AutoCloseable

Throws:

IOException