alluxio.multi.process

Class MultiProcessCluster

java.lang.Object

alluxio.multi.process.MultiProcessCluster

@ThreadSafe
public final class MultiProcessCluster
extends Object

Class for starting, stopping, and interacting with an Alluxio cluster where each master and worker runs in its own process. Compared to LocalAlluxioCluster, MultiProcessCluster is - Slower - Black box testing only (No access to master/worker internals) - Destructible (You can kill -9 masters/workers) - More realistic of real deployments Due to the slower speed, [@link LocalAlluxioCluster} should generally be preferred. MultiProcessCluster is primarily for tests which want to stop or restart servers. The synchronization strategy for this class is to synchronize all public methods.

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static class	MultiProcessCluster.Builder Builder for MultiProcessCluster.
static class	MultiProcessCluster.DeployMode Deploy mode for the cluster.

Field Summary

Fields

Modifier and Type	Field and Description
static String	ALLUXIO_USE_FIXED_TEST_PORTS
static int	PORTS_PER_MASTER
static int	PORTS_PER_WORKER

Method Summary

Modifier and Type	Method and Description
void	addExternalMasterAddress(MasterNetAddress externalMasterAddress) Updates internal master list with an external address.
void	destroy() Destroys the cluster.
void	formatJournal() Formats the cluster journal.
Clients	getClients()
MultiProcessCluster.DeployMode	getDeployMode()
FileSystem	getFileSystemClient()
FileSystemContext	getFilesystemContext()
String	getJournalDir()
String	getJournalDir(int masterId)
JournalMasterClient	getJournalMasterClientForMaster()
List<MasterNetAddress>	getMasterAddresses()
MasterInquireClient	getMasterInquireClient()
MetaMasterClient	getMetaMasterClient()
MetricsMasterClient	getMetricsMasterClient()
int	getPrimaryMasterIndex(int timeoutMs) Gets the index of the primary master.
static MultiProcessCluster.Builder	newBuilder(List<PortCoordination.ReservedPort> reservedPorts)
void	notifySuccess() Informs the cluster that the test succeeded.
void	restartZk() Restarts the Zookeeper cluster.
void	saveWorkdir() Copies the work directory to the artifacts folder.
void	start() Starts the cluster, launching all server processes.
void	startMaster(int i) Starts the specified master.
void	startMasters() Starts all masters.
void	startNewMasters(int count, boolean format) Start a number of new master nodes.
void	startWorker(int i) Starts the specified worker.
void	stopAndRemoveMaster(int i) Removes master i from the cluster.
void	stopMaster(int i)
void	stopMasters() Stops all masters.
void	stopWorker(int i)
void	stopZk() Stops the Zookeeper cluster.
void	updateDeployMode(MultiProcessCluster.DeployMode mode) Updates the cluster's deploy mode.
void	updateMasterConf(PropertyKey key, String value) Updates master configuration for all masters.
void	waitForAllNodesRegistered(int timeoutMs) Waits for all nodes to be registered.
int	waitForAndKillPrimaryMaster(int timeoutMs) Kills the primary master.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

ALLUXIO_USE_FIXED_TEST_PORTS

public static final String ALLUXIO_USE_FIXED_TEST_PORTS

See Also:

Constant Field Values

PORTS_PER_MASTER

public static final int PORTS_PER_MASTER

See Also:

Constant Field Values

PORTS_PER_WORKER

public static final int PORTS_PER_WORKER

See Also:

Constant Field Values

Method Detail

start

public void start()
           throws Exception

Starts the cluster, launching all server processes.

Throws:

Exception

startNewMasters

public void startNewMasters(int count,
                            boolean format)
                     throws Exception

Start a number of new master nodes.

Parameters:

count - the count of the masters to start

format - whether to format the master first

Throws:

Exception - if any error occurs

waitForAndKillPrimaryMaster

public int waitForAndKillPrimaryMaster(int timeoutMs)
                                throws TimeoutException,
                                       InterruptedException

Kills the primary master. If no master is currently primary, this method blocks until a primary has been elected, then kills it.

Parameters:

timeoutMs - maximum amount of time to wait, in milliseconds

Returns:

the ID of the killed master

Throws:

TimeoutException

InterruptedException

getPrimaryMasterIndex

public int getPrimaryMasterIndex(int timeoutMs)
                          throws TimeoutException,
                                 InterruptedException

Gets the index of the primary master.

Parameters:

timeoutMs - maximum amount of time to wait, in milliseconds

Returns:

the index of the primary master

Throws:

TimeoutException

InterruptedException

waitForAllNodesRegistered

public void waitForAllNodesRegistered(int timeoutMs)
                               throws TimeoutException,
                                      InterruptedException

Waits for all nodes to be registered.

Parameters:

timeoutMs - maximum amount of time to wait, in milliseconds

Throws:

TimeoutException

InterruptedException

getDeployMode

public MultiProcessCluster.DeployMode getDeployMode()

Returns:

the deploy mode

getFilesystemContext

public FileSystemContext getFilesystemContext()

Returns:

the FileSystemContext which can be used to access the cluster

getFileSystemClient

public FileSystem getFileSystemClient()

Returns:

a client for interacting with the cluster

getMetaMasterClient

public MetaMasterClient getMetaMasterClient()

Returns:

a meta master client

getMetricsMasterClient

public MetricsMasterClient getMetricsMasterClient()

Returns:

a metrics master client

getJournalMasterClientForMaster

public JournalMasterClient getJournalMasterClientForMaster()

Returns:

a journal master client for alluxio master

getClients

public Clients getClients()

Returns:

clients for communicating with the cluster

notifySuccess

public void notifySuccess()

Informs the cluster that the test succeeded. If this method is never called, the cluster will save a copy of its state during teardown.

saveWorkdir

public void saveWorkdir()
                 throws IOException

Copies the work directory to the artifacts folder.

Throws:

IOException

destroy

public void destroy()
             throws IOException

Destroys the cluster. It may not be re-started after being destroyed.

Throws:

IOException

startMasters

public void startMasters()

Starts all masters.

startMaster

public void startMaster(int i)
                 throws IOException

Starts the specified master.

Parameters:

i - the index of the master to start

Throws:

IOException

startWorker

public void startWorker(int i)
                 throws IOException

Starts the specified worker.

Parameters:

i - the index of the worker to start

Throws:

IOException

stopMasters

public void stopMasters()

Stops all masters.

stopMaster

public void stopMaster(int i)
                throws IOException

Parameters:

i - the index of the master to stop

Throws:

IOException

stopAndRemoveMaster

public void stopAndRemoveMaster(int i)
                         throws IOException

Removes master i from the cluster.

Parameters:

i - index of the master to be removed

Throws:

IOException

updateMasterConf

public void updateMasterConf(PropertyKey key,
                             @Nullable
                             String value)

Updates master configuration for all masters. This will take effect on a master the next time the master is started.

Parameters:

key - the key to update

value - the value to set, or null to unset the key

addExternalMasterAddress

public void addExternalMasterAddress(MasterNetAddress externalMasterAddress)

Updates internal master list with an external address. This API is to provide support when this cluster is joined by external masters. Calling this API with an external master address will make this cluster aware of it when creating clients/contexts etc.

Parameters:

externalMasterAddress - external master address

updateDeployMode

public void updateDeployMode(MultiProcessCluster.DeployMode mode)

Updates the cluster's deploy mode.

Parameters:

mode - the mode to set

stopWorker

public void stopWorker(int i)
                throws IOException

Parameters:

i - the index of the worker to stop

Throws:

IOException

getJournalDir

public String getJournalDir()

Returns:

the journal directory

getJournalDir

public String getJournalDir(int masterId)

Parameters:

masterId - index of the master

Returns:

the journal directory for the specified master

getMasterAddresses

public List<MasterNetAddress> getMasterAddresses()

Returns:

return the list of master addresses

stopZk

public void stopZk()
            throws IOException

Stops the Zookeeper cluster.

Throws:

IOException

restartZk

public void restartZk()
               throws Exception

Restarts the Zookeeper cluster.

Throws:

Exception

formatJournal

public void formatJournal()
                   throws IOException

Formats the cluster journal.

Throws:

IOException

getMasterInquireClient

public MasterInquireClient getMasterInquireClient()

Returns:

a client for determining the serving master

newBuilder

public static MultiProcessCluster.Builder newBuilder(List<PortCoordination.ReservedPort> reservedPorts)

Parameters:

reservedPorts - ports reserved for usage by this cluster

Returns:

a new builder for an MultiProcessCluster