alluxio.hadoop

Class FileSystem

java.lang.Object

org.apache.hadoop.conf.Configured

org.apache.hadoop.fs.FileSystem

alluxio.hadoop.AbstractFileSystem

alluxio.hadoop.FileSystem

All Implemented Interfaces:

Closeable, AutoCloseable, org.apache.hadoop.conf.Configurable, org.apache.hadoop.fs.PathCapabilities, org.apache.hadoop.security.token.DelegationTokenIssuer

@NotThreadSafe
public class FileSystem
extends AbstractFileSystem

An Alluxio client API compatible with Apache Hadoop FileSystem interface. Any program working with Hadoop HDFS can work with Alluxio transparently. Note that the performance of using this API may not be as efficient as the performance of using the Alluxio native API defined in FileSystem, which this API is built on top of.

Nested Class Summary

Nested classes/interfaces inherited from class org.apache.hadoop.fs.FileSystem

org.apache.hadoop.fs.FileSystem.DirectoryEntries, org.apache.hadoop.fs.FileSystem.DirListingIterator<T extends org.apache.hadoop.fs.FileStatus>, org.apache.hadoop.fs.FileSystem.Statistics

Field Summary

Fields inherited from class alluxio.hadoop.AbstractFileSystem

mAlluxioConf, mFileSystem

Fields inherited from class org.apache.hadoop.fs.FileSystem

DEFAULT_FS, FS_DEFAULT_NAME_KEY, SHUTDOWN_HOOK_PRIORITY, statistics, TRASH_PREFIX, USER_HOME_PREFIX

Constructor Summary

Constructors

Constructor and Description
FileSystem() Constructs a new FileSystem.
FileSystem(FileSystem fileSystem) Constructs a new FileSystem instance with a specified FileSystem handler for tests.

Method Summary

Modifier and Type	Method and Description
protected AlluxioURI	getAlluxioPath(org.apache.hadoop.fs.Path path) Used to convert hadoop path to Alluxio path.
protected Map<String,Object>	getConfigurationFromUri(URI uri, org.apache.hadoop.conf.Configuration conf) Gets the connection configuration from the input uri.
protected org.apache.hadoop.fs.Path	getFsPath(String fsUriHeader, URIStatus fileStatus) Used to get FS native path from Alluxio file status.
protected String	getFsScheme(URI fsUri) Used to get FS scheme.
String	getScheme() Gets the URI scheme that maps to the FileSystem.
protected boolean	isZookeeperMode()
protected void	validateFsUri(URI fsUri) Validates given FS base URI for scheme and authority.

Methods inherited from class alluxio.hadoop.AbstractFileSystem

access, append, canonicalizeUri, close, create, createNonRecursive, delete, delete, getCanonicalServiceName, getDefaultBlockSize, getDefaultPort, getDefaultReplication, getFileBlockLocations, getFileStatus, getUri, getWorkingDirectory, initialize, initialize, listStatus, mkdirs, open, rename, setOwner, setPermission, setReplication, setWorkingDirectory

Methods inherited from class org.apache.hadoop.fs.FileSystem

append, append, appendFile, areSymlinksEnabled, cancelDeleteOnExit, checkPath, clearStatistics, closeAll, closeAllForUGI, completeLocalOutput, concat, copyFromLocalFile, copyFromLocalFile, copyFromLocalFile, copyFromLocalFile, copyToLocalFile, copyToLocalFile, copyToLocalFile, create, create, create, create, create, create, create, create, create, create, create, create, createDataInputStreamBuilder, createDataInputStreamBuilder, createDataOutputStreamBuilder, createFile, createMultipartUploader, createNewFile, createNonRecursive, createNonRecursive, createPathHandle, createSnapshot, createSnapshot, createSymlink, deleteOnExit, deleteSnapshot, enableSymlinks, exists, fixRelativePart, get, get, get, getAclStatus, getAdditionalTokenIssuers, getAllStatistics, getAllStoragePolicies, getBlockSize, getCanonicalUri, getChildFileSystems, getContentSummary, getDefaultBlockSize, getDefaultReplication, getDefaultUri, getDelegationToken, getFileBlockLocations, getFileChecksum, getFileChecksum, getFileLinkStatus, getFileSystemClass, getFSofPath, getGlobalStorageStatistics, getHomeDirectory, getInitialWorkingDirectory, getLength, getLinkTarget, getLocal, getName, getNamed, getPathHandle, getQuotaUsage, getReplication, getServerDefaults, getServerDefaults, getStatistics, getStatistics, getStatus, getStatus, getStoragePolicy, getStorageStatistics, getTrashRoot, getTrashRoots, getUsed, getUsed, getXAttr, getXAttrs, getXAttrs, globStatus, globStatus, hasPathCapability, isDirectory, isFile, listCorruptFileBlocks, listFiles, listLocatedStatus, listLocatedStatus, listStatus, listStatus, listStatus, listStatusBatch, listStatusIterator, listXAttrs, makeQualified, mkdirs, mkdirs, modifyAclEntries, moveFromLocalFile, moveFromLocalFile, moveToLocalFile, msync, newInstance, newInstance, newInstance, newInstanceLocal, open, open, open, openFile, openFile, openFileWithOptions, openFileWithOptions, primitiveCreate, primitiveMkdir, primitiveMkdir, printStatistics, processDeleteOnExit, removeAcl, removeAclEntries, removeDefaultAcl, removeXAttr, rename, renameSnapshot, resolveLink, resolvePath, satisfyStoragePolicy, setAcl, setDefaultUri, setDefaultUri, setQuota, setQuotaByStorageType, setStoragePolicy, setTimes, setVerifyChecksum, setWriteChecksum, setXAttr, setXAttr, startLocalOutput, supportsSymlinks, truncate, unsetStoragePolicy

Methods inherited from class org.apache.hadoop.conf.Configured

getConf, setConf

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface org.apache.hadoop.security.token.DelegationTokenIssuer

addDelegationTokens, collectDelegationTokens

Constructor Detail

FileSystem

public FileSystem()

Constructs a new FileSystem.

FileSystem

public FileSystem(FileSystem fileSystem)

Constructs a new FileSystem instance with a specified FileSystem handler for tests.

Parameters:

fileSystem - handler to file system

Method Detail

getScheme

public String getScheme()

Description copied from class: AbstractFileSystem

Gets the URI scheme that maps to the FileSystem. This was introduced in Hadoop 2.x as a means to make loading new FileSystem s simpler.

Specified by:

getScheme in class AbstractFileSystem

Returns:

scheme hadoop should map to

See Also:

FileSystem.createFileSystem(java.net.URI, org.apache.hadoop.conf.Configuration)

isZookeeperMode

protected boolean isZookeeperMode()

Specified by:

isZookeeperMode in class AbstractFileSystem

Returns:

true if zookeeper should be used

getConfigurationFromUri

protected Map<String,Object> getConfigurationFromUri(URI uri,
                                                     org.apache.hadoop.conf.Configuration conf)

Description copied from class: AbstractFileSystem

Gets the connection configuration from the input uri.

Specified by:

getConfigurationFromUri in class AbstractFileSystem

Parameters:

uri - an Alluxio Uri that may contain connection configuration

validateFsUri

protected void validateFsUri(URI fsUri)
                      throws IOException,
                             IllegalArgumentException

Description copied from class: AbstractFileSystem

Validates given FS base URI for scheme and authority.

Specified by:

validateFsUri in class AbstractFileSystem

Parameters:

fsUri - FS Uri

Throws:

IOException

IllegalArgumentException

getFsScheme

protected String getFsScheme(URI fsUri)

Description copied from class: AbstractFileSystem

Used to get FS scheme.

Specified by:

getFsScheme in class AbstractFileSystem

Parameters:

fsUri - file system base URI

Returns:

file system scheme

getAlluxioPath

protected AlluxioURI getAlluxioPath(org.apache.hadoop.fs.Path path)

Description copied from class: AbstractFileSystem

Used to convert hadoop path to Alluxio path.

Specified by:

getAlluxioPath in class AbstractFileSystem

Parameters:

path - the input path

Returns:

the Alluxio path

getFsPath

protected org.apache.hadoop.fs.Path getFsPath(String fsUriHeader,
                                              URIStatus fileStatus)

Description copied from class: AbstractFileSystem

Used to get FS native path from Alluxio file status.

Specified by:

getFsPath in class AbstractFileSystem

Parameters:

fsUriHeader - FS URI header -> "scheme://authority"

fileStatus - Alluxio file status

Returns:

FS native path