alluxio.master.lineage

Interface LineageMaster

All Superinterfaces:

JournalEntryIterable, JournalEntryStateMachine, Master, Server<Boolean>

All Known Implementing Classes:

DefaultLineageMaster

public interface LineageMaster
extends Master

The lineage master interface stores the lineage metadata in Alluxio, this interface contains common components that manage all lineage-related activities.

Method Summary

Modifier and Type	Method and Description
long	createLineage(List<AlluxioURI> inputFiles, List<AlluxioURI> outputFiles, Job job) Creates a lineage.
boolean	deleteLineage(long lineageId, boolean cascade) Deletes a lineage.
List<LineageInfo>	getLineageInfoList()
LineageStoreView	getLineageStoreView()
long	reinitializeFile(String path, long blockSizeBytes, long ttl, TtlAction ttlAction) Reinitializes the file when the file is lost or not completed.
void	reportLostFile(String path) Reports a file as lost.
void	scheduleCheckpoint(CheckpointPlan plan) Schedules persistence for the output files of the given checkpoint plan.

Methods inherited from interface alluxio.master.Master

createJournalContext

Methods inherited from interface alluxio.master.journal.JournalEntryStateMachine

getName, processJournalEntry, resetState

Methods inherited from interface alluxio.master.journal.JournalEntryIterable

getJournalEntryIterator

Methods inherited from interface alluxio.Server

getDependencies, getName, getServices, start, stop

Method Detail

getLineageStoreView

LineageStoreView getLineageStoreView()

Returns:

a lineage store view wrapping the contained lineage store

createLineage

long createLineage(List<AlluxioURI> inputFiles,
                   List<AlluxioURI> outputFiles,
                   Job job)
            throws InvalidPathException,
                   FileAlreadyExistsException,
                   BlockInfoException,
                   IOException,
                   AccessControlException,
                   FileDoesNotExistException

Creates a lineage. It creates a new file for each output file.

Parameters:

inputFiles - the input files

outputFiles - the output files

job - the job

Returns:

the id of the created lineage

Throws:

InvalidPathException - if the path to the input file is invalid

FileAlreadyExistsException - if the output file already exists

BlockInfoException - if fails to create the output file

AccessControlException - if the permission check fails

FileDoesNotExistException - if any of the input files do not exist

IOException

deleteLineage

boolean deleteLineage(long lineageId,
                      boolean cascade)
               throws LineageDoesNotExistException,
                      LineageDeletionException,
                      UnavailableException

Deletes a lineage.

Parameters:

lineageId - id the of lineage

cascade - the flag if to delete all the downstream lineages

Returns:

true if the lineage is deleted, false otherwise

Throws:

LineageDoesNotExistException - the lineage does not exist

LineageDeletionException - the lineage deletion fails

UnavailableException

reinitializeFile

long reinitializeFile(String path,
                      long blockSizeBytes,
                      long ttl,
                      TtlAction ttlAction)
               throws InvalidPathException,
                      LineageDoesNotExistException,
                      AccessControlException,
                      FileDoesNotExistException,
                      UnavailableException

Reinitializes the file when the file is lost or not completed.

Parameters:

path - the path to the file

blockSizeBytes - the block size

ttl - the TTL

ttlAction - action to perform on ttl expiry

Returns:

the id of the reinitialized file when the file is lost or not completed, -1 otherwise

Throws:

InvalidPathException - the file path is invalid

LineageDoesNotExistException - when the file does not exist

AccessControlException - if permission checking fails

FileDoesNotExistException - if the path does not exist

UnavailableException

getLineageInfoList

List<LineageInfo> getLineageInfoList()
                              throws LineageDoesNotExistException,
                                     FileDoesNotExistException

Returns:

the list of all the LineageInfos

Throws:

LineageDoesNotExistException - if the lineage does not exist

FileDoesNotExistException - if any associated file does not exist

scheduleCheckpoint

void scheduleCheckpoint(CheckpointPlan plan)

Schedules persistence for the output files of the given checkpoint plan.

Parameters:

plan - the plan for checkpointing

reportLostFile

void reportLostFile(String path)
             throws FileDoesNotExistException,
                    AccessControlException,
                    InvalidPathException,
                    UnavailableException

Reports a file as lost.

Parameters:

path - the path to the file

Throws:

FileDoesNotExistException - if the file does not exist

AccessControlException - if permission checking fails

InvalidPathException - if the path is invalid

UnavailableException