alluxio.master.file.meta

Class UfsSyncPathCache

java.lang.Object

alluxio.master.file.meta.UfsSyncPathCache

@ThreadSafe
public class UfsSyncPathCache
extends Object

A cache of paths that have been synchronized or have been marked as not being synchronized with the UFS. Paths are mapped to time values that indicate their last synchronization or invalidation. Synchronization times are updated only at the root path for which a sync was performed by InodeSyncStream. Each path contains the following values concerning synchronization: - recursiveSyncTime: the last time a sync was successful on this path - directChildSyncTime: the last time a sync was successful on the path which (at least) included itself and its direct children - recursiveSyncTime: the last time a sync was successful on this path which included itself and all its children Each path additionally contains the following values considering invalidation: - invalidationTime: the last time this exact path was invalidated - directChildInvalidation: the last time a direct child of this path was invalidated - recursiveChildInvalidation: the last time a non-direct child of this path was invalidated - isFile: when checking if a file needs to be synced, any descendant type from the most recent sync will be valid Whenever an invalidation is received the path, and its parents up to the root have their appropriate invalidation times updated. Validation times are updated on the path when notifySyncedPath(alluxio.AlluxioURI, alluxio.file.options.DescendantType, long, java.lang.Long, boolean) is called on the root sync path after a successful sync. An invalidation is received either when a client calls DefaultFileSystemMaster.needsSync(alluxio.AlluxioURI) to notify that a path needs synchronization, or when a file is updated by an external Alluxio cluster, and cross cluster sync is enabled. Checking if a path needs to be synchronized involves checking the appropriate validation and invalidation times at each path component up to the root.

Field Summary

Fields

Modifier and Type	Field and Description
com.google.common.cache.Cache<String,alluxio.master.file.meta.SyncState>	mItems

Constructor Summary

Constructors

Constructor and Description
UfsSyncPathCache(java.time.Clock clock) Creates a new instance of UfsSyncPathCache.

Method Summary

Modifier and Type	Method and Description
Optional<Pair<Long,Long>>	getSyncTimesForPath(AlluxioURI path) Get sync times for a given path if they exist in the cache.
void	notifyInvalidation(AlluxioURI path) Notify that a path has been invalidated.
alluxio.master.file.meta.SyncState	notifySyncedPath(AlluxioURI path, DescendantType descendantType, long startTime, Long syncTime, boolean isFile) Notify sync happened.
long	recordStartSync() Called when starting a sync.
SyncCheck	shouldSyncPath(AlluxioURI path, long intervalMs, DescendantType descendantType) Check if sync should happen.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

mItems

public final com.google.common.cache.Cache<String,alluxio.master.file.meta.SyncState> mItems

Constructor Detail

UfsSyncPathCache

public UfsSyncPathCache(java.time.Clock clock)

Creates a new instance of UfsSyncPathCache.

Parameters:

clock - the clock to use to compute sync times

Method Detail

recordStartSync

public long recordStartSync()

Called when starting a sync.

Returns:

the time at the start of the sync

getSyncTimesForPath

public Optional<Pair<Long,Long>> getSyncTimesForPath(AlluxioURI path)

Get sync times for a given path if they exist in the cache.

Parameters:

path - the path to check

Returns:

a pair of sync times, where element 0 is the direct sync time and element 1 is the recursive sync time

shouldSyncPath

public SyncCheck shouldSyncPath(AlluxioURI path,
                                long intervalMs,
                                DescendantType descendantType)
                         throws InvalidPathException

Check if sync should happen. A path is checked starting from the full path, all the way up to the root. At each path the largest validation time and invalidation time is computed depending on its descendant type. After each path component is checked, if the last invalidation time is more recent than the last validation time then a sync is needed. Otherwise, a sync is needed based on the difference between the current time and the last sync time and the interval.

Parameters:

path - the path to check

intervalMs - the frequency in ms that the sync should happen

descendantType - the descendant type of the operation being performed

Returns:

a SyncCheck object indicating if a sync should be performed

Throws:

InvalidPathException

notifyInvalidation

public void notifyInvalidation(AlluxioURI path)
                        throws InvalidPathException

Notify that a path has been invalidated.

Parameters:

path - the path

Throws:

InvalidPathException

notifySyncedPath

public alluxio.master.file.meta.SyncState notifySyncedPath(AlluxioURI path,
                                                           DescendantType descendantType,
                                                           long startTime,
                                                           @Nullable
                                                           Long syncTime,
                                                           boolean isFile)

Notify sync happened.

Parameters:

path - the synced path

descendantType - the descendant type for the performed operation

startTime - the time at which the sync was started

syncTime - the time to set the sync success to, if null then the current clock time is used

isFile - true if the synced path is a file

Returns:

the sync state