alluxio.master.lineage

Class DefaultLineageMaster

java.lang.Object

alluxio.master.AbstractMaster

alluxio.master.lineage.DefaultLineageMaster

All Implemented Interfaces:

JournalEntryIterable, JournalEntryStateMachine, LineageMaster, Master, Server<Boolean>

@NotThreadSafe
public final class DefaultLineageMaster
extends AbstractMaster
implements LineageMaster

The default lineage master stores the lineage metadata in Alluxio, and it contains the components that manage all lineage-related activities.

Field Summary

Fields inherited from class alluxio.master.AbstractMaster

mBackupManager, mClock, mPauseStateLock, mSafeModeManager, mStateChangeLock

Method Summary

Modifier and Type	Method and Description
long	createLineage(List<AlluxioURI> inputFiles, List<AlluxioURI> outputFiles, Job job) Creates a lineage.
boolean	deleteLineage(long lineageId, boolean cascade) Deletes a lineage.
Set<Class<? extends Server>>	getDependencies()
Iterator<Journal.JournalEntry>	getJournalEntryIterator()
List<LineageInfo>	getLineageInfoList()
LineageStoreView	getLineageStoreView()
String	getName()
Map<String,org.apache.thrift.TProcessor>	getServices()
void	processJournalEntry(Journal.JournalEntry entry) Applies a journal entry to the state machine.
long	reinitializeFile(String path, long blockSizeBytes, long ttl, TtlAction ttlAction) Reinitializes the file when the file is lost or not completed.
void	reportLostFile(String path) Reports a file as lost.
void	resetState() Resets the journaled internal state of the state machine.
void	scheduleCheckpoint(CheckpointPlan plan) Schedules persistence for the output files of the given checkpoint plan.
void	start(Boolean isLeader) Starts the Alluxio server.

Methods inherited from class alluxio.master.AbstractMaster

createJournalContext, getExecutorService, stop

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface alluxio.master.Master

createJournalContext

Methods inherited from interface alluxio.Server

stop

Method Detail

getServices

public Map<String,org.apache.thrift.TProcessor> getServices()

Specified by:

getServices in interface Server<Boolean>

Returns:

a map from service names to TProcessors that serve RPCs for this server

getName

public String getName()

Specified by:

getName in interface JournalEntryStateMachine

Specified by:

getName in interface Server<Boolean>

Returns:

the name of this journal entry state machine

getDependencies

public Set<Class<? extends Server>> getDependencies()

Specified by:

getDependencies in interface Server<Boolean>

Overrides:

getDependencies in class AbstractMaster

Returns:

the server dependencies

processJournalEntry

public void processJournalEntry(Journal.JournalEntry entry)
                         throws IOException

Description copied from interface: JournalEntryStateMachine

Applies a journal entry to the state machine.

Specified by:

processJournalEntry in interface JournalEntryStateMachine

Parameters:

entry - the entry to process to update the state of the state machine

Throws:

IOException

resetState

public void resetState()

Description copied from interface: JournalEntryStateMachine

Resets the journaled internal state of the state machine.

Specified by:

resetState in interface JournalEntryStateMachine

start

public void start(Boolean isLeader)
           throws IOException

Description copied from interface: Server

Starts the Alluxio server.

Specified by:

start in interface Server<Boolean>

Overrides:

start in class AbstractMaster

Parameters:

isLeader - the start options

Throws:

IOException

getJournalEntryIterator

public Iterator<Journal.JournalEntry> getJournalEntryIterator()

Specified by:

getJournalEntryIterator in interface JournalEntryIterable

Returns:

an Iterator that iterates all the journal entries

getLineageStoreView

public LineageStoreView getLineageStoreView()

Specified by:

getLineageStoreView in interface LineageMaster

Returns:

a lineage store view wrapping the contained lineage store

createLineage

public long createLineage(List<AlluxioURI> inputFiles,
                          List<AlluxioURI> outputFiles,
                          Job job)
                   throws InvalidPathException,
                          FileAlreadyExistsException,
                          BlockInfoException,
                          IOException,
                          AccessControlException,
                          FileDoesNotExistException

Description copied from interface: LineageMaster

Creates a lineage. It creates a new file for each output file.

Specified by:

createLineage in interface LineageMaster

Parameters:

inputFiles - the input files

outputFiles - the output files

job - the job

Returns:

the id of the created lineage

Throws:

InvalidPathException - if the path to the input file is invalid

FileAlreadyExistsException - if the output file already exists

BlockInfoException - if fails to create the output file

AccessControlException - if the permission check fails

FileDoesNotExistException - if any of the input files do not exist

IOException

deleteLineage

public boolean deleteLineage(long lineageId,
                             boolean cascade)
                      throws LineageDoesNotExistException,
                             LineageDeletionException,
                             UnavailableException

Description copied from interface: LineageMaster

Deletes a lineage.

Specified by:

deleteLineage in interface LineageMaster

Parameters:

lineageId - id the of lineage

cascade - the flag if to delete all the downstream lineages

Returns:

true if the lineage is deleted, false otherwise

Throws:

LineageDoesNotExistException - the lineage does not exist

LineageDeletionException - the lineage deletion fails

UnavailableException

reinitializeFile

public long reinitializeFile(String path,
                             long blockSizeBytes,
                             long ttl,
                             TtlAction ttlAction)
                      throws InvalidPathException,
                             LineageDoesNotExistException,
                             AccessControlException,
                             FileDoesNotExistException,
                             UnavailableException

Description copied from interface: LineageMaster

Reinitializes the file when the file is lost or not completed.

Specified by:

reinitializeFile in interface LineageMaster

Parameters:

path - the path to the file

blockSizeBytes - the block size

ttl - the TTL

ttlAction - action to perform on ttl expiry

Returns:

the id of the reinitialized file when the file is lost or not completed, -1 otherwise

Throws:

InvalidPathException - the file path is invalid

LineageDoesNotExistException - when the file does not exist

AccessControlException - if permission checking fails

FileDoesNotExistException - if the path does not exist

UnavailableException

getLineageInfoList

public List<LineageInfo> getLineageInfoList()
                                     throws LineageDoesNotExistException,
                                            FileDoesNotExistException

Specified by:

getLineageInfoList in interface LineageMaster

Returns:

the list of all the LineageInfos

Throws:

LineageDoesNotExistException - if the lineage does not exist

FileDoesNotExistException - if any associated file does not exist

scheduleCheckpoint

public void scheduleCheckpoint(CheckpointPlan plan)

Description copied from interface: LineageMaster

Schedules persistence for the output files of the given checkpoint plan.

Specified by:

scheduleCheckpoint in interface LineageMaster

Parameters:

plan - the plan for checkpointing

reportLostFile

public void reportLostFile(String path)
                    throws FileDoesNotExistException,
                           AccessControlException,
                           InvalidPathException,
                           UnavailableException

Description copied from interface: LineageMaster

Reports a file as lost.

Specified by:

reportLostFile in interface LineageMaster

Parameters:

path - the path to the file

Throws:

FileDoesNotExistException - if the file does not exist

AccessControlException - if permission checking fails

InvalidPathException - if the path is invalid

UnavailableException