alluxio.underfs.adl

Class AdlUnderFileSystem

java.lang.Object

alluxio.underfs.BaseUnderFileSystem

alluxio.underfs.ConsistentUnderFileSystem

alluxio.underfs.hdfs.HdfsUnderFileSystem

alluxio.underfs.adl.AdlUnderFileSystem

All Implemented Interfaces:

AtomicFileOutputStreamCallback, UnderFileSystem, Closeable, AutoCloseable

@ThreadSafe
public class AdlUnderFileSystem
extends HdfsUnderFileSystem

An UnderFileSystem uses the Microsoft Azure Data Lake Storage.

Nested Class Summary

Nested classes/interfaces inherited from interface alluxio.underfs.UnderFileSystem

UnderFileSystem.Factory, UnderFileSystem.SpaceType

Field Summary

Fields

Modifier and Type	Field and Description
static String	SCHEME_INSECURE Constant for the adl URI scheme.
static String	SCHEME_SECURE Constant for the adls URI scheme.

Fields inherited from class alluxio.underfs.BaseUnderFileSystem

mUfsConf, mUri

Constructor Summary

Constructors

Constructor and Description
AdlUnderFileSystem(AlluxioURI ufsUri, UnderFileSystemConfiguration conf, org.apache.hadoop.conf.Configuration adlConf) Constructs a new Adl UnderFileSystem.

Method Summary

Modifier and Type	Method and Description
static org.apache.hadoop.conf.Configuration	createConfiguration(UnderFileSystemConfiguration conf) Prepares the configuration for this Adl as an HDFS configuration.
static AdlUnderFileSystem	createInstance(AlluxioURI uri, UnderFileSystemConfiguration conf) Factory method to construct a new Adl UnderFileSystem.
long	getBlockSizeByte(String path) Gets the block size of a file in under file system, in bytes.
List<String>	getFileLocations(String path) Gets the list of locations of the indicated path.
List<String>	getFileLocations(String path, FileLocationOptions options) Gets the list of locations of the indicated path given options.
UfsStatus	getStatus(String path) Gets the file or directory status.
String	getUnderFSType() Returns the name of the under filesystem implementation.
void	setMode(String path, short mode) Changes posix file mode.
void	setOwner(String path, String user, String group) Sets the user and group of the given path.

Methods inherited from class alluxio.underfs.hdfs.HdfsUnderFileSystem

cleanup, close, connectFromMaster, connectFromWorker, create, createDirect, deleteDirectory, deleteFile, exists, getAclPair, getActiveSyncInfo, getDirectoryStatus, getFileStatus, getSpace, isDirectory, isFile, isSeekable, listStatus, mkdirs, open, renameDirectory, renameFile, setAclEntries, startActiveSyncPolling, startSync, stopActiveSyncPolling, stopSync, supportsActiveSync, supportsFlush

Methods inherited from class alluxio.underfs.ConsistentUnderFileSystem

createNonexistingFile, createNonexistingFile, deleteExistingDirectory, deleteExistingDirectory, deleteExistingFile, getExistingDirectoryStatus, getExistingFileStatus, getExistingStatus, isExistingDirectory, openExistingFile, openExistingFile, renameRenamableDirectory, renameRenamableFile

Methods inherited from class alluxio.underfs.BaseUnderFileSystem

create, deleteDirectory, getConfiguration, getFingerprint, getOperationMode, getPhysicalStores, isObjectStorage, listStatus, mkdirs, open, resolveUri, validatePath

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface alluxio.underfs.UnderFileSystem

create, createNonexistingFile, createNonexistingFile, deleteDirectory, deleteExistingDirectory, deleteExistingDirectory, deleteExistingFile, getConfiguration, getExistingDirectoryStatus, getExistingFileStatus, getExistingStatus, getFingerprint, getOperationMode, getPhysicalStores, isExistingDirectory, isObjectStorage, listStatus, mkdirs, open, openExistingFile, openExistingFile, renameRenamableDirectory, renameRenamableFile, resolveUri

Field Detail

SCHEME_INSECURE

public static final String SCHEME_INSECURE

Constant for the adl URI scheme.

See Also:

Constant Field Values

SCHEME_SECURE

public static final String SCHEME_SECURE

Constant for the adls URI scheme.

See Also:

Constant Field Values

Constructor Detail

AdlUnderFileSystem

public AdlUnderFileSystem(AlluxioURI ufsUri,
                          UnderFileSystemConfiguration conf,
                          org.apache.hadoop.conf.Configuration adlConf)

Constructs a new Adl UnderFileSystem.

Parameters:

ufsUri - the AlluxioURI for this UFS

conf - the configuration for this UFS

adlConf - the configuration for this Adl UFS

Method Detail

createConfiguration

public static org.apache.hadoop.conf.Configuration createConfiguration(UnderFileSystemConfiguration conf)

Prepares the configuration for this Adl as an HDFS configuration.

Parameters:

conf - the configuration for this UFS

Returns:

the created configuration

createInstance

public static AdlUnderFileSystem createInstance(AlluxioURI uri,
                                                UnderFileSystemConfiguration conf)

Factory method to construct a new Adl UnderFileSystem.

Parameters:

uri - the AlluxioURI for this UFS

conf - the configuration for this UFS

Returns:

a new Adl UnderFileSystem instance

getUnderFSType

public String getUnderFSType()

Description copied from interface: UnderFileSystem

Returns the name of the under filesystem implementation. The name should be lowercase and not include any spaces, e.g. "hdfs", "s3".

Specified by:

getUnderFSType in interface UnderFileSystem

Overrides:

getUnderFSType in class HdfsUnderFileSystem

Returns:

name of the under filesystem implementation

getBlockSizeByte

public long getBlockSizeByte(String path)
                      throws IOException

Description copied from interface: UnderFileSystem

Gets the block size of a file in under file system, in bytes.

Specified by:

getBlockSizeByte in interface UnderFileSystem

Overrides:

getBlockSizeByte in class HdfsUnderFileSystem

Parameters:

path - the file name

Returns:

the block size in bytes

Throws:

IOException

getStatus

public UfsStatus getStatus(String path)
                    throws IOException

Description copied from interface: UnderFileSystem

Gets the file or directory status. The caller does not need to know if the path is a file or directory. This method will determine the path type, and will return the appropriate status.

Specified by:

getStatus in interface UnderFileSystem

Overrides:

getStatus in class HdfsUnderFileSystem

Parameters:

path - the path to get the status

Returns:

the file or directory status

Throws:

FileNotFoundException - when the path does not exist

IOException

setOwner

public void setOwner(String path,
                     String user,
                     String group)

Description copied from interface: UnderFileSystem

Sets the user and group of the given path. An empty implementation should be provided if unsupported.

Specified by:

setOwner in interface UnderFileSystem

Overrides:

setOwner in class HdfsUnderFileSystem

Parameters:

path - the path of the file

user - the new owner to set, unchanged if null

group - the new group to set, unchanged if null

setMode

public void setMode(String path,
                    short mode)

Description copied from interface: UnderFileSystem

Changes posix file mode.

Specified by:

setMode in interface UnderFileSystem

Overrides:

setMode in class HdfsUnderFileSystem

Parameters:

path - the path of the file

mode - the mode to set in short format, e.g. 0777

getFileLocations

public List<String> getFileLocations(String path)
                              throws IOException

Description copied from interface: UnderFileSystem

Gets the list of locations of the indicated path.

Specified by:

getFileLocations in interface UnderFileSystem

Overrides:

getFileLocations in class HdfsUnderFileSystem

Parameters:

path - the file name

Returns:

The list of locations

Throws:

IOException

getFileLocations

public List<String> getFileLocations(String path,
                                     FileLocationOptions options)
                              throws IOException

Description copied from interface: UnderFileSystem

Gets the list of locations of the indicated path given options.

Specified by:

getFileLocations in interface UnderFileSystem

Overrides:

getFileLocations in class HdfsUnderFileSystem

Parameters:

path - the file name

options - method options

Returns:

The list of locations

Throws:

IOException