alluxio.master.file.activesync

Class ActiveSyncManager

java.lang.Object

alluxio.master.file.activesync.ActiveSyncManager

All Implemented Interfaces:

Checkpointed, Journaled, JournalEntryIterable

@NotThreadSafe
public class ActiveSyncManager
extends Object
implements Journaled

Manager for the Active UFS sync process. There are several threads cooperating to make the active sync process happen. 1. An active polling thread that polls HDFS for change events and aggregates these events. 2. A heartbeat thread that wakes up periodically to consume the aggregated events, and perform syncing if necessary. 3. For initial syncing, we launch a future to perform initial syncing asynchronously. This is stored in mSyncPathStatus.

Constructor Summary

Constructors

Constructor and Description
ActiveSyncManager(MountTable mountTable, FileSystemMaster fileSystemMaster) Constructs a Active Sync Manager.

Method Summary

Modifier and Type	Method and Description
void	applyAndJournal(java.util.function.Supplier<JournalContext> context, File.AddSyncPointEntry entry) Apply File.AddSyncPointEntry and journal the entry.
void	applyAndJournal(java.util.function.Supplier<JournalContext> context, File.RemoveSyncPointEntry entry) Apply File.RemoveSyncPointEntry and journal the entry.
CheckpointName	getCheckpointName()
ExecutorService	getExecutor() Get SyncManager Executor.
List<AlluxioURI>	getFilterList(long mountId) Get the filter list associated with mount Id.
CloseableIterator<Journal.JournalEntry>	getJournalEntryIterator()
Lock	getLock() Gets the lock protecting the syncManager.
RetryPolicy	getRetryPolicy() Gets the retry policy.
List<SyncPointInfo>	getSyncPathList() Get the sync point list.
boolean	isSyncPoint(AlluxioURI syncPoint)
boolean	isUnderSyncPoint(AlluxioURI path) Check if a URI is actively synced.
void	launchPollingThread(long mountId, long txId) Launches polling thread on a particular mount point with starting txId.
boolean	processJournalEntry(Journal.JournalEntry entry) Attempts to apply a journal entry.
void	recoverFromStartSync(AlluxioURI uri, long mountId) Recover from start sync operation.
void	recoverFromStopSync(AlluxioURI uri) Recover from a stop sync operation.
void	resetState() Resets the object's journaled state.
void	setTxId(long mountId, long txId) Set the transaction id for a particular mountId.
void	start() Start the polling threads.
void	startSyncAndJournal(RpcContext rpcContext, AlluxioURI syncPoint) Start active sync on a URI and journal the add entry.
void	stop() Stops the sync manager and any outstanding threads, does not change the sync points.
void	stopSyncAndJournal(RpcContext rpcContext, AlluxioURI syncPoint) Stop active sync on a URI and journal the remove entry.
void	stopSyncForMount(long mountId) Stop active sync on a mount id.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface alluxio.master.journal.Journaled

applyAndJournal, restoreFromCheckpoint, writeToCheckpoint

Constructor Detail

ActiveSyncManager

public ActiveSyncManager(MountTable mountTable,
                         FileSystemMaster fileSystemMaster)

Constructs a Active Sync Manager.

Parameters:

mountTable - mount table

fileSystemMaster - file system master

Method Detail

getLock

public Lock getLock()

Gets the lock protecting the syncManager.

Returns:

syncmanager lock

getRetryPolicy

public RetryPolicy getRetryPolicy()

Gets the retry policy.

Returns:

retry policy

isSyncPoint

public boolean isSyncPoint(AlluxioURI syncPoint)

Parameters:

syncPoint - the uri to check

Returns:

true if the given path is a sync point

isUnderSyncPoint

public boolean isUnderSyncPoint(AlluxioURI path)

Check if a URI is actively synced.

Parameters:

path - path to check

Returns:

true if a URI is being actively synced

start

public void start()
           throws IOException

Start the polling threads.

Throws:

IOException

launchPollingThread

public void launchPollingThread(long mountId,
                                long txId)

Launches polling thread on a particular mount point with starting txId.

Parameters:

mountId - launch polling thread on a mount id

txId - specifies the transaction id to initialize the pollling thread

applyAndJournal

public void applyAndJournal(java.util.function.Supplier<JournalContext> context,
                            File.AddSyncPointEntry entry)

Apply File.AddSyncPointEntry and journal the entry.

Parameters:

context - journal context

entry - addSyncPoint entry

applyAndJournal

public void applyAndJournal(java.util.function.Supplier<JournalContext> context,
                            File.RemoveSyncPointEntry entry)

Apply File.RemoveSyncPointEntry and journal the entry.

Parameters:

context - journal context

entry - removeSyncPoint entry

startSyncAndJournal

public void startSyncAndJournal(RpcContext rpcContext,
                                AlluxioURI syncPoint)
                         throws InvalidPathException

Start active sync on a URI and journal the add entry.

Parameters:

rpcContext - the master rpc or no-op context

syncPoint - sync point to be start

Throws:

InvalidPathException

stopSyncForMount

public void stopSyncForMount(long mountId)
                      throws InvalidPathException

Stop active sync on a mount id.

Parameters:

mountId - mountId to stop active sync

Throws:

InvalidPathException

stopSyncAndJournal

public void stopSyncAndJournal(RpcContext rpcContext,
                               AlluxioURI syncPoint)
                        throws InvalidPathException

Stop active sync on a URI and journal the remove entry.

Parameters:

rpcContext - the master rpc or no-op context

syncPoint - sync point to be stopped

Throws:

InvalidPathException

getFilterList

public List<AlluxioURI> getFilterList(long mountId)

Get the filter list associated with mount Id.

Parameters:

mountId - mountId

Returns:

a list of URIs (sync points) associated with that mount id

getSyncPathList

public List<SyncPointInfo> getSyncPathList()

Get the sync point list.

Returns:

a list of URIs (sync points)

setTxId

public void setTxId(long mountId,
                    long txId)

Set the transaction id for a particular mountId.

Parameters:

mountId - mount id

txId - transaction id

getExecutor

public ExecutorService getExecutor()

Get SyncManager Executor.

Returns:

an executor for active syncing

stop

public void stop()

Stops the sync manager and any outstanding threads, does not change the sync points. This stops four things in the following order. 1. Stop any outstanding initial sync futures for the sync points. (syncFuture.cancel) 2. Stop the heartbeat thread that periodically wakes up to process events that have been recorded for the past heartbeat interval. 3. Tell the polling thread to stop monitoring the path for events 4. Stop the thread that is polling HDFS for events

recoverFromStopSync

public void recoverFromStopSync(AlluxioURI uri)

Recover from a stop sync operation.

Parameters:

uri - uri to stop sync

recoverFromStartSync

public void recoverFromStartSync(AlluxioURI uri,
                                 long mountId)

Recover from start sync operation.

Parameters:

uri - uri to start sync

mountId - mount id of the uri

processJournalEntry

public boolean processJournalEntry(Journal.JournalEntry entry)

Description copied from interface: Journaled

Attempts to apply a journal entry.

Specified by:

processJournalEntry in interface Journaled

Parameters:

entry - the entry to apply

Returns:

whether the entry type is supported by this journaled object

resetState

public void resetState()

Resets the object's journaled state. It clears all sync points, and stops the polling thread.

Specified by:

resetState in interface Journaled

getCheckpointName

public CheckpointName getCheckpointName()

Specified by:

getCheckpointName in interface Checkpointed

Returns:

a name for this checkpointed class

getJournalEntryIterator

public CloseableIterator<Journal.JournalEntry> getJournalEntryIterator()

Specified by:

getJournalEntryIterator in interface JournalEntryIterable

Returns:

an CloseableIterator that iterates all the journal entries