alluxio.underfs

Class UfsStatusCache

java.lang.Object

alluxio.underfs.UfsStatusCache

@ThreadSafe
public class UfsStatusCache
extends Object

This class is a cache from an Alluxio namespace URI (AlluxioURI, i.e. /path/to/inode) to UFS statuses. It also allows associating a path with child inodes, so that the statuses for a specific path can be searched for later.

Constructor Summary

Constructors

Constructor and Description
UfsStatusCache(ExecutorService prefetchExecutor, UfsAbsentPathCache absentPathCache, long cacheValidTime) Create a new instance of UfsStatusCache.

Method Summary

Modifier and Type	Method and Description
Collection<UfsStatus>	addChildren(AlluxioURI path, Collection<UfsStatus> children) Add a parent-child mapping to the status cache.
UfsStatus	addStatus(AlluxioURI path, UfsStatus status) Add a new status to the cache.
void	cancelAllPrefetch() Interrupts and cancels any currently running prefetch jobs.
Collection<UfsStatus>	fetchChildrenIfAbsent(RpcContext rpcContext, AlluxioURI path, MountTable mountTable) Fetches children of a given alluxio path stores them in the cache, then returns them.
Collection<UfsStatus>	fetchChildrenIfAbsent(RpcContext rpcContext, AlluxioURI path, MountTable mountTable, boolean useFallback) Fetches children of a given alluxio path, stores them in the cache, then returns them.
UfsStatus	fetchStatusIfAbsent(AlluxioURI path, MountTable mountTable) Attempts to return a status from the cache.
Collection<UfsStatus>	getChildren(AlluxioURI path) Get the child UfsStatuses from a given AlluxioURI.
UfsStatus	getStatus(AlluxioURI path) Get the UfsStatus from a given AlluxioURI.
Optional<UfsStatus>	hasStatus(AlluxioURI path) Check if a status has been loaded in the cache for the given AlluxioURI.
Future<Collection<UfsStatus>>	prefetchChildren(AlluxioURI path, MountTable mountTable) Submit a request to asynchronously fetch the statuses corresponding to a given directory.
UfsStatus	remove(AlluxioURI path) Remove a status from the cache.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

UfsStatusCache

public UfsStatusCache(@Nullable
                      ExecutorService prefetchExecutor,
                      UfsAbsentPathCache absentPathCache,
                      long cacheValidTime)

Create a new instance of UfsStatusCache.

Parameters:

prefetchExecutor - the executor service used to prefetch statuses. If set to null, then calls to prefetchChildren(AlluxioURI, MountTable) will not schedule any tasks.

absentPathCache - the absent cache that ufsStatusCache should consult

cacheValidTime - the time when the absent cache entry would be considered valid

Method Detail

addStatus

@Nullable
public UfsStatus addStatus(AlluxioURI path,
                                     UfsStatus status)

Add a new status to the cache. The last component of the path in the AlluxioURI must match the result of UfsStatus.getName(). This method overrides any status currently cached for the same URI.

Parameters:

path - the Alluxio path to key on

status - the ufs status to store

Returns:

the previous status for the path if it existed, null otherwise

Throws:

IllegalArgumentException - if the status name doesn't match the final URI path component

addChildren

@Nullable
public Collection<UfsStatus> addChildren(AlluxioURI path,
                                                   Collection<UfsStatus> children)

Add a parent-child mapping to the status cache. All child statuses added via this method will be available via getStatus(AlluxioURI).

Parameters:

path - the directory inode path which contains the children

children - the children of the path

Returns:

the previous set of children if the mapping existed, null otherwise

remove

@Nullable
public UfsStatus remove(AlluxioURI path)

Remove a status from the cache. This will remove any references to child UfsStatus.

Parameters:

path - the path corresponding to the UfsStatus to remove

Returns:

the removed UfsStatus

getStatus

@Nullable
public UfsStatus getStatus(AlluxioURI path)
                              throws FileNotFoundException

Get the UfsStatus from a given AlluxioURI.

Parameters:

path - the path the retrieve

Returns:

The corresponding UfsStatus or null if there is none stored

Throws:

FileNotFoundException - if the UFS does not contain the file

hasStatus

public Optional<UfsStatus> hasStatus(AlluxioURI path)

Check if a status has been loaded in the cache for the given AlluxioURI. Same as getStatus(AlluxioURI) except does not check the absent cache.

Parameters:

path - the path the retrieve

Returns:

The corresponding UfsStatus

fetchStatusIfAbsent

@Nullable
public UfsStatus fetchStatusIfAbsent(AlluxioURI path,
                                               MountTable mountTable)
                                        throws InvalidPathException

Attempts to return a status from the cache. If it doesn't exist, reaches to the UFS for it.

Parameters:

path - the path the retrieve

mountTable - the Alluxio mount table

Returns:

The corresponding UfsStatus or null if there is none stored

Throws:

InvalidPathException

fetchChildrenIfAbsent

@Nullable
public Collection<UfsStatus> fetchChildrenIfAbsent(RpcContext rpcContext,
                                                             AlluxioURI path,
                                                             MountTable mountTable,
                                                             boolean useFallback)
                                                      throws InterruptedException,
                                                             InvalidPathException

Fetches children of a given alluxio path, stores them in the cache, then returns them. Children can be returned in a few ways 1. Children already exist in the internal index. We simply return them 2. If children did not already exist in the index, then check if there was a scheduled prefetch job running for this path. If so, wait for the job to finish and return the result. 3. If no prefetch job, and children don't yet exist in the cache, then if the fallback parameter is true, fetch them from the UFS and store them in the cache. Otherwise, simply return null.

Parameters:

rpcContext - the rpcContext of the source of this call

path - the Alluxio path to get the children of

mountTable - the Alluxio mount table

useFallback - whether or not to fall back to calling the UFS

Returns:

child UFS statuses of the alluxio path, or null if no prefetch job and fallback specified as false

Throws:

InvalidPathException - if the alluxio path can't be resolved to a UFS mount

InterruptedException

fetchChildrenIfAbsent

@Nullable
public Collection<UfsStatus> fetchChildrenIfAbsent(RpcContext rpcContext,
                                                             AlluxioURI path,
                                                             MountTable mountTable)
                                                      throws InterruptedException,
                                                             InvalidPathException

Fetches children of a given alluxio path stores them in the cache, then returns them. Will always return statuses from the UFS whether or not they exist in the cache, and whether a prefetch job was scheduled or not.

Parameters:

rpcContext - the rpcContext of the source of this call

path - the Alluxio path

mountTable - the Alluxio mount table

Returns:

child UFS statuses of the alluxio path

Throws:

InvalidPathException - if the alluxio path can't be resolved to a UFS mount

InterruptedException

getChildren

@Nullable
public Collection<UfsStatus> getChildren(AlluxioURI path)

Get the child UfsStatuses from a given AlluxioURI.

Parameters:

path - the path the retrieve

Returns:

The corresponding UfsStatus or null if there is none stored

prefetchChildren

@Nullable
public Future<Collection<UfsStatus>> prefetchChildren(AlluxioURI path,
                                                                MountTable mountTable)

Submit a request to asynchronously fetch the statuses corresponding to a given directory. Retrieve any fetched statuses by calling fetchChildrenIfAbsent(RpcContext, AlluxioURI, MountTable) with the same Alluxio path. If no ExecutorService was provided to this object before instantiation, this method is a no-op.

Parameters:

path - the path to prefetch

mountTable - the Alluxio mount table

Returns:

the future corresponding to the fetch task

cancelAllPrefetch

public void cancelAllPrefetch()

Interrupts and cancels any currently running prefetch jobs.