alluxio.underfs

Class AbstractUfsManager

java.lang.Object

alluxio.underfs.AbstractUfsManager

All Implemented Interfaces:

UfsManager, Closeable, AutoCloseable

Direct Known Subclasses:

DoraUfsManager, JobUfsManager, MasterUfsManager, NoopUfsManager, WorkerUfsManager

public abstract class AbstractUfsManager
extends Object
implements UfsManager

Basic implementation of UfsManager. Store the journal UFS and root mount point information.

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static class	AbstractUfsManager.Key The key of the UFS cache.

Nested classes/interfaces inherited from interface alluxio.underfs.UfsManager

UfsManager.UfsClient

Field Summary

Fields

Modifier and Type	Field and Description
protected com.google.common.io.Closer	mCloser
protected ConcurrentHashMap<AbstractUfsManager.Key,UnderFileSystem>	mUnderFileSystemMap Maps from key to UnderFileSystem instances.

Constructor Summary

Constructors

Modifier	Constructor and Description
protected	AbstractUfsManager()

Method Summary

Modifier and Type	Method and Description
void	addMount(long mountId, AlluxioURI ufsUri, UnderFileSystemConfiguration ufsConf) Keeps track of a mount id and maps it to its URI in Alluxio and configuration.
void	addMountWithRecorder(long mountId, AlluxioURI ufsUri, UnderFileSystemConfiguration ufsConf, Recorder recorder) Keeps track of a mount id and maps it to its URI in Alluxio and configuration and records the execution process.
void	close()
protected abstract void	connectUfs(UnderFileSystem fs) Takes any necessary actions required to establish a connection to the under file system.
protected AbstractUfsManager.Key	generateKey(AlluxioURI ufsUri, UnderFileSystemConfiguration ufsConf)
protected Optional<UnderFileSystem>	get(AbstractUfsManager.Key key)
Optional<UnderFileSystem>	get(AlluxioURI ufsUri, UnderFileSystemConfiguration ufsConf) Gets an instance for the given UFS URI and configuration, if such exists.
UfsManager.UfsClient	get(long mountId) Gets UFS information from the manager if this mount ID exists, or throws exception otherwise.
UfsManager.UfsClient	getJournal(URI location)
UnderFileSystem	getOrAdd(AlluxioURI ufsUri, UnderFileSystemConfiguration ufsConf) Return a UFS instance if it already exists in the cache, otherwise, creates a new instance and return it.
UfsManager.UfsClient	getRoot()
boolean	hasMount(long mountId) Checks if a mountId exists.
void	removeMount(long mountId) Removes the association from a mount id to a UFS instance.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

mUnderFileSystemMap

protected final ConcurrentHashMap<AbstractUfsManager.Key,UnderFileSystem> mUnderFileSystemMap

Maps from key to UnderFileSystem instances. This map keeps the entire set of UFS instances, each keyed by their unique combination of Uri and conf information. This map helps efficiently identify if a UFS instance in request should be created or can be reused.

mCloser

protected final com.google.common.io.Closer mCloser

Constructor Detail

AbstractUfsManager

protected AbstractUfsManager()

Method Detail

getOrAdd

public UnderFileSystem getOrAdd(AlluxioURI ufsUri,
                                UnderFileSystemConfiguration ufsConf)

Return a UFS instance if it already exists in the cache, otherwise, creates a new instance and return it.

Parameters:

ufsUri - the UFS path

ufsConf - the UFS configuration

Returns:

the UFS instance

generateKey

protected AbstractUfsManager.Key generateKey(AlluxioURI ufsUri,
                                             UnderFileSystemConfiguration ufsConf)

connectUfs

protected abstract void connectUfs(UnderFileSystem fs)
                            throws IOException

Takes any necessary actions required to establish a connection to the under file system. The implementation will either call UnderFileSystem.connectFromMaster(String) or UnderFileSystem.connectFromWorker(String) depending on the running process.

Throws:

IOException

addMount

public void addMount(long mountId,
                     AlluxioURI ufsUri,
                     UnderFileSystemConfiguration ufsConf)

Description copied from interface: UfsManager

Keeps track of a mount id and maps it to its URI in Alluxio and configuration. This is an Alluxio-only operation and no interaction to UFS will be made.

Specified by:

addMount in interface UfsManager

Parameters:

mountId - the mount id

ufsUri - the UFS path

ufsConf - the UFS configuration

addMountWithRecorder

public void addMountWithRecorder(long mountId,
                                 AlluxioURI ufsUri,
                                 UnderFileSystemConfiguration ufsConf,
                                 Recorder recorder)

Description copied from interface: UfsManager

Keeps track of a mount id and maps it to its URI in Alluxio and configuration and records the execution process. This is an Alluxio-only operation and no interaction to UFS will be made.

Specified by:

addMountWithRecorder in interface UfsManager

Parameters:

mountId - the mount id

ufsUri - the UFS path

ufsConf - the UFS configuration

recorder - recorder used to record the detailed execution process

removeMount

public void removeMount(long mountId)

Description copied from interface: UfsManager

Removes the association from a mount id to a UFS instance. If the mount id is not known, this is a noop.

Specified by:

removeMount in interface UfsManager

Parameters:

mountId - the mount id

get

public Optional<UnderFileSystem> get(AlluxioURI ufsUri,
                                     UnderFileSystemConfiguration ufsConf)

Gets an instance for the given UFS URI and configuration, if such exists.

Parameters:

ufsUri - the URI of the UFS

ufsConf - the configuration for the UFS

Returns:

a UFS instance, or none if not registered

get

protected Optional<UnderFileSystem> get(AbstractUfsManager.Key key)

get

public UfsManager.UfsClient get(long mountId)
                         throws NotFoundException,
                                UnavailableException

Description copied from interface: UfsManager

Gets UFS information from the manager if this mount ID exists, or throws exception otherwise.

Specified by:

get in interface UfsManager

Parameters:

mountId - the mount id

Returns:

the UFS information

Throws:

NotFoundException - if mount id is not found in mount table

UnavailableException - if master is not available to query for mount table

getRoot

public UfsManager.UfsClient getRoot()

Specified by:

getRoot in interface UfsManager

Returns:

the UFS client associated with root

getJournal

public UfsManager.UfsClient getJournal(URI location)

Specified by:

getJournal in interface UfsManager

Parameters:

location - the journal location

Returns:

the UFS client associated with journal location

close

public void close()
           throws IOException

Specified by:

close in interface Closeable

Specified by:

close in interface AutoCloseable

Throws:

IOException

hasMount

public boolean hasMount(long mountId)

Description copied from interface: UfsManager

Checks if a mountId exists.

Specified by:

hasMount in interface UfsManager

Parameters:

mountId - the mountId to check

Returns:

whether the mountId has been occupied