alluxio.master.journal.raft

Class SnapshotReplicationManager

java.lang.Object

alluxio.master.journal.raft.SnapshotReplicationManager

public class SnapshotReplicationManager
extends Object

Class for managing snapshot replication among masters. It manages two snapshot replication workflows - worker to master and master to worker. 1. Worker to Master When a raft leader needs a snapshot, instead of taking snapshot locally it copies a recent snapshot from one of the followers. Workflow: - Ratis calls leader state machine to take a snapshot - leader gets snapshot metadata from follower - leader pick one of the follower and send a request for copying the snapshot - follower receives the request and calls the leader raft journal service to upload the snapshot - after the upload completes, leader remembers the temporary snapshot location and index - Ratis calls the leader state machine again to take a snapshot - leader moves the temporary snapshot to the journal snapshot folder and returns snapshot index 2. Master to Worker When a raft follower receives a notification to download a snapshot, it downloads the latest snapshot from the leader. Workflow: - Ratis leader determines one of the follower needs a snapshot because it misses journal entries from a long time ago - Ratis leader notifies Ratis follower to install a snapshot from leader, the follower calls the Alluxio state machine to fulfill this request - the follower state machine calls the snapshot manager which calls the raft journal service from leader to download a snapshot - after the downloads completes, follower moves the file to snapshot directory and gives Ratis the snapshot index

Constructor Summary

Constructors

Constructor and Description
SnapshotReplicationManager(RaftJournalSystem journalSystem, org.apache.ratis.statemachine.impl.SimpleStateMachineStorage storage)

Method Summary

Modifier and Type	Method and Description
org.apache.ratis.protocol.Message	handleRequest(JournalQueryRequest queryRequest) Handles snapshot requests.
CompletableFuture<org.apache.ratis.server.protocol.TermIndex>	installSnapshotFromLeader() Downloads and installs a snapshot from the leader.
long	maybeCopySnapshotFromFollower() Attempts to copy a snapshot from one of the followers.
io.grpc.stub.StreamObserver<UploadSnapshotPRequest>	receiveSnapshotFromFollower(io.grpc.stub.StreamObserver<UploadSnapshotPResponse> responseStreamObserver) Receives a snapshot from follower.
io.grpc.stub.StreamObserver<DownloadSnapshotPRequest>	sendSnapshotToFollower(io.grpc.stub.StreamObserver<DownloadSnapshotPResponse> responseObserver) Sends a snapshot to a follower.
void	sendSnapshotToLeader() Sends a snapshot to the leader.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

SnapshotReplicationManager

public SnapshotReplicationManager(RaftJournalSystem journalSystem,
                                  org.apache.ratis.statemachine.impl.SimpleStateMachineStorage storage)

Parameters:

journalSystem - the raft journal system

storage - the snapshot storage

Method Detail

installSnapshotFromLeader

public CompletableFuture<org.apache.ratis.server.protocol.TermIndex> installSnapshotFromLeader()

Downloads and installs a snapshot from the leader.

Returns:

a future with the term index of the installed snapshot

sendSnapshotToLeader

public void sendSnapshotToLeader()
                          throws IOException

Sends a snapshot to the leader.

Throws:

IOException - if error occurs while initializing the data stream

maybeCopySnapshotFromFollower

public long maybeCopySnapshotFromFollower()

Attempts to copy a snapshot from one of the followers. The leader state machine calls this method regularly when it needs a new snapshot. To avoid blocking normal journal operations, This method always returns a value immediately without waiting for download to finish: - If no download is in progress, it schedules a new download asynchronously and returns RaftLog.INVALID_LOG_INDEX. - If a download is in progress, it returns RaftLog.INVALID_LOG_INDEX immediately. - If a download is completed, it moves the downloaded file to the snapshot directory and returns the snapshot index.

Returns:

the index of the downloaded snapshot, or RaftLog.INVALID_LOG_INDEX if no snapshot is installed.

receiveSnapshotFromFollower

public io.grpc.stub.StreamObserver<UploadSnapshotPRequest> receiveSnapshotFromFollower(io.grpc.stub.StreamObserver<UploadSnapshotPResponse> responseStreamObserver)

Receives a snapshot from follower.

Parameters:

responseStreamObserver - the response stream observer

Returns:

the request stream observer

handleRequest

public org.apache.ratis.protocol.Message handleRequest(JournalQueryRequest queryRequest)
                                                throws IOException

Handles snapshot requests.

Parameters:

queryRequest - the query request

Returns:

the response message, or null if the request is not handled

Throws:

IOException - if any error occurred while handling the request

sendSnapshotToFollower

public io.grpc.stub.StreamObserver<DownloadSnapshotPRequest> sendSnapshotToFollower(io.grpc.stub.StreamObserver<DownloadSnapshotPResponse> responseObserver)

Sends a snapshot to a follower.

Parameters:

responseObserver - the response stream observer

Returns:

the request stream observer