alluxio.master.journal

Interface JournalSystem

All Known Implementing Classes:

AbstractJournalSystem, NoopJournalSystem, RaftJournalSystem, UfsJournalSystem

@ThreadSafe
public interface JournalSystem

A journal system for storing and applying journal entries. To use the journal system, first create per-state-machine journals with the createJournal(Master) method. Once all state machines are added, start() the journal system. The journal system starts in standby mode, meaning it will not accept writes, but will apply journal entries to keep state machine states up to date with previously written journal entries. To enable writes, the journal system may be changed to primary mode, where all state machines will first be caught up fully with the latest journal updates, then begin to accept writes. Note that as a performance optimization, the journal system does not apply journal entries to state machines while in primary mode. Instead, the state machine states must be directly modified by RPC handlers. The journal system may also be changed from primary to standby mode. This transition is done by resetting all state machines and re-building them by catching up on the journal. Example usage:

 JournalSystem journalSystem = new JournalSystemImpl();
 if (!journalSystem.isFormatted()) {
   journalSystem.format();
 }
 Journal blockMasterJournal = journalSystem.createJournal(blockMaster);
 Journal fileSystemMasterJournal = journalSystem.createJournal(fileSystemMaster);

 // The journal system always starts in standby mode. It must be transitioned to primary mode
 // before it can write entries.
 journalSystem.start();
 journalSystem.setPrimary(true);

 try (JournalContext c = blockMasterJournal.createJournalContext()) {
   c.append(exampleBlockJournalEntry);
 }
 // At this point, the journal entry is persistently committed to the journal and will be applied
 // asynchronously to the in-memory state of all standby masters.
 try (JournalContext c = fileSystemMasterJournal.createJournalContext()) {
   c.append(exampleFileSystemJournalEntry);
 }
 // Transition to a standby journal. In this mode, the journal will apply entries to the masters
 // as they are committed to the log.
 journalSystem.setPrimary(false);
 

Nested Class Summary

Nested Classes

Modifier and Type	Interface and Description
static class	JournalSystem.Builder Builder for constructing a journal system.
static class	JournalSystem.Mode The mode of the journal system.

Method Summary

Modifier and Type	Method and Description
void	addJournalSink(Master master, JournalSink journalSink)
CatchupFuture	catchup(Map<String,Long> journalSequenceNumbers) Initiates a catching up of journals to given sequences.
void	checkpoint() Creates a checkpoint in the primary master journal system.
Journal	createJournal(Master master) Creates a journal for the given state machine.
void	format() Formats the journal system.
void	gainPrimacy() Transitions the journal to primary mode.
Map<String,Long>	getCurrentSequenceNumbers() Used to get the current state from a leader journal system.
default Map<ServiceType,GrpcService>	getJournalServices()
Set<JournalSink>	getJournalSinks(Master master)
boolean	isEmpty() Returns whether the journal is formatted and has not had any entries written to it yet.
boolean	isFormatted()
void	losePrimacy() Transitions the journal to standby mode.
void	removeJournalSink(Master master, JournalSink journalSink)
void	resume() Resumes applying for all journals.
void	start() Starts the journal system.
void	stop() Stops the journal system.
void	suspend(Runnable interruptCallback) Suspends applying for all journals.
default void	waitForCatchup() Waits for the journal catchup to finish when the process starts.

Method Detail

createJournal

Journal createJournal(Master master)

Creates a journal for the given state machine. The returned journal can create journal contexts for writing journal entries. However, no entries may be written until the journal system has been started, and entries may only be written when the journal system is in PRIMARY mode. When the journal is started in standby mode, it will call Journaled#processJournalEntry(JournalEntry) and Journaled.resetState() to keep the state machine's state in sync with the entries written to the journal.

Parameters:

master - the master to create the journal for

Returns:

a new instance of Journal

start

void start()
    throws InterruptedException,
           IOException

Starts the journal system. All journals must be created before starting the journal system. This method will block until the journal system is successfully started. The journal always starts in standby mode.

Throws:

InterruptedException

IOException

stop

void stop()
   throws InterruptedException,
          IOException

Stops the journal system.

Throws:

InterruptedException

IOException

gainPrimacy

void gainPrimacy()

Transitions the journal to primary mode.

losePrimacy

void losePrimacy()

Transitions the journal to standby mode.

suspend

void suspend(Runnable interruptCallback)
      throws IOException

Suspends applying for all journals. When using suspend, the caller needs to provide a callback method as parameter. This callback is invoked when the journal needs to reload and thus cannot suspend the state changes any more. The callback should cancel any tasks that access the master states. After the callback returns, the journal assumes that the states is no longer being accessed and will reload immediately.

Parameters:

interruptCallback - the callback function to be invoked when the suspension is interrupted

Throws:

IOException

resume

void resume()
     throws IOException

Resumes applying for all journals. Note: Journal system should have been suspended prior to calling this.

Throws:

IOException

catchup

CatchupFuture catchup(Map<String,Long> journalSequenceNumbers)
               throws IOException

Initiates a catching up of journals to given sequences. Note: Journal system should have been suspended prior to calling this.

Parameters:

journalSequenceNumbers - sequence to advance per each journal

Returns:

the future to track when catching up is completed

Throws:

IOException

waitForCatchup

default void waitForCatchup()

Waits for the journal catchup to finish when the process starts. This is intended to be only be called when starting the Alluxio master process in standby mode and before the standby master becoming the primary master. This is best-effort, because even if it did not finish replaying the journal, the rest of the system will still complete the journal catchup in a different phase. This can be implemented by a journal type to optimize the journal replay, and avoid getting interrupted with primary state changes during journal replay.

getCurrentSequenceNumbers

Map<String,Long> getCurrentSequenceNumbers()

Used to get the current state from a leader journal system. Note: State changes to journals must have been effectively blocked by a state-lock before calling this method.

Returns:

the current map of sequences for each master

format

void format()
     throws IOException

Formats the journal system.

Throws:

IOException

isFormatted

boolean isFormatted()
             throws IOException

Returns:

whether the journal system has been formatted

Throws:

IOException

addJournalSink

void addJournalSink(Master master,
                    JournalSink journalSink)

Parameters:

master - the master for which to add the journal sink

journalSink - the journal sink to add

removeJournalSink

void removeJournalSink(Master master,
                       JournalSink journalSink)

Parameters:

master - the master from which to remove the journal sink

journalSink - the journal sink to remove

getJournalSinks

Set<JournalSink> getJournalSinks(@Nullable
                                 Master master)

Parameters:

master - the master to get the journal sinks for, or null to get all sinks

Returns:

a set of JournalSink for the given master, or all sinks if master is null

isEmpty

boolean isEmpty()

Returns whether the journal is formatted and has not had any entries written to it yet. This can only be determined when the journal system is in primary mode because entries are written to the primary first.

Returns:

whether the journal system is freshly formatted

checkpoint

void checkpoint()
         throws IOException

Creates a checkpoint in the primary master journal system.

Throws:

IOException

getJournalServices

default Map<ServiceType,GrpcService> getJournalServices()

Returns:

RPC services for journal system