alluxio.worker.block

Class BlockMasterClient

java.lang.Object

alluxio.AbstractClient

alluxio.AbstractMasterClient

alluxio.worker.block.BlockMasterClient

All Implemented Interfaces:

Client, Closeable, AutoCloseable

Direct Known Subclasses:

CachingBlockMasterClient

@ThreadSafe
public class BlockMasterClient
extends AbstractMasterClient

A wrapper for the gRPC client to interact with the block master, used by alluxio worker.

Nested Class Summary

Nested classes/interfaces inherited from class alluxio.AbstractClient

AbstractClient.RpcCallable<V>

Field Summary

Fields inherited from class alluxio.AbstractClient

mAddress, mChannel, mClosed, mConnected, mContext, mServiceVersion, mVersionService

Constructor Summary

Constructors

Constructor and Description
BlockMasterClient(MasterClientContext conf) Creates a new instance of BlockMasterClient for the worker.

Method Summary

Modifier and Type	Method and Description
protected void	afterConnect() This method is called after the connection is made to the remote.
void	commitBlock(long workerId, long usedBytesOnTier, String tierAlias, String mediumType, long blockId, long length) Commits a block on a worker.
void	commitBlockInUfs(long blockId, long length) Commits a block in Ufs.
List<LocationBlockIdListEntry>	convertBlockListMapToProto(Map<BlockStoreLocation,List<Long>> blockListOnLocation) Converts the block list map to a proto list.
long	getId(WorkerNetAddress address) Returns a worker id for a workers net address.
protected ServiceType	getRemoteServiceType()
protected String	getServiceName()
protected long	getServiceVersion()
Command	heartbeat(long workerId, Map<String,Long> capacityBytesOnTiers, Map<String,Long> usedBytesOnTiers, List<Long> removedBlocks, Map<BlockStoreLocation,List<Long>> addedBlocks, Map<String,List<String>> lostStorage, List<Metric> metrics) The method the worker should periodically execute to heartbeat back to the master.
void	register(long workerId, List<String> storageTierAliases, Map<String,Long> totalBytesOnTiers, Map<String,Long> usedBytesOnTiers, Map<BlockStoreLocation,List<Long>> currentBlocksOnLocation, Map<String,List<String>> lostStorage, List<ConfigProperty> configList) The method the worker should execute to register with the block master.

Methods inherited from class alluxio.AbstractMasterClient

getAddress, getConfAddress

Methods inherited from class alluxio.AbstractClient

afterDisconnect, beforeConnect, beforeDisconnect, checkVersion, close, connect, disconnect, getRemoteServiceVersion, isClosed, isConnected, retryRPC, retryRPC

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

BlockMasterClient

public BlockMasterClient(MasterClientContext conf)

Creates a new instance of BlockMasterClient for the worker.

Parameters:

conf - master client configuration

Method Detail

getRemoteServiceType

protected ServiceType getRemoteServiceType()

Specified by:

getRemoteServiceType in class AbstractClient

Returns:

the type of remote service

getServiceName

protected String getServiceName()

Specified by:

getServiceName in class AbstractClient

Returns:

a string representing the specific service

getServiceVersion

protected long getServiceVersion()

Specified by:

getServiceVersion in class AbstractClient

Returns:

the client service version

afterConnect

protected void afterConnect()
                     throws IOException

Description copied from class: AbstractClient

This method is called after the connection is made to the remote. Implementations should create internal state to finish the connection process.

Overrides:

afterConnect in class AbstractClient

Throws:

IOException

commitBlock

public void commitBlock(long workerId,
                        long usedBytesOnTier,
                        String tierAlias,
                        String mediumType,
                        long blockId,
                        long length)
                 throws IOException

Commits a block on a worker.

Parameters:

workerId - the worker id committing the block

usedBytesOnTier - the amount of used bytes on the tier the block is committing to

tierAlias - the alias of the tier the block is being committed to

mediumType - the medium type the block is being committed to

blockId - the block id being committed

length - the length of the block being committed

Throws:

IOException

commitBlockInUfs

public void commitBlockInUfs(long blockId,
                             long length)
                      throws IOException

Commits a block in Ufs.

Parameters:

blockId - the block id being committed

length - the length of the block being committed

Throws:

IOException

getId

public long getId(WorkerNetAddress address)
           throws IOException

Returns a worker id for a workers net address.

Parameters:

address - the net address to get a worker id for

Returns:

a worker id

Throws:

IOException

convertBlockListMapToProto

public List<LocationBlockIdListEntry> convertBlockListMapToProto(Map<BlockStoreLocation,List<Long>> blockListOnLocation)

Converts the block list map to a proto list. Because the list is flattened from a map, in the list no two LocationBlockIdListEntry instances shall have the same BlockStoreLocationProto. The uniqueness of BlockStoreLocationProto is determined by tier alias and medium type. That means directories with the same tier alias and medium type will be merged into the same LocationBlockIdListEntry.

Parameters:

blockListOnLocation - a map from block location to the block list

Returns:

a flattened and deduplicated list

heartbeat

public Command heartbeat(long workerId,
                         Map<String,Long> capacityBytesOnTiers,
                         Map<String,Long> usedBytesOnTiers,
                         List<Long> removedBlocks,
                         Map<BlockStoreLocation,List<Long>> addedBlocks,
                         Map<String,List<String>> lostStorage,
                         List<Metric> metrics)
                  throws IOException

The method the worker should periodically execute to heartbeat back to the master.

Parameters:

workerId - the worker id

capacityBytesOnTiers - a mapping from storage tier alias to capacity bytes

usedBytesOnTiers - a mapping from storage tier alias to used bytes

removedBlocks - a list of block removed from this worker

addedBlocks - a mapping from storage tier alias to added blocks

lostStorage - a mapping from storage tier alias to a list of lost storage paths

metrics - a list of worker metrics

Returns:

an optional command for the worker to execute

Throws:

IOException

register

public void register(long workerId,
                     List<String> storageTierAliases,
                     Map<String,Long> totalBytesOnTiers,
                     Map<String,Long> usedBytesOnTiers,
                     Map<BlockStoreLocation,List<Long>> currentBlocksOnLocation,
                     Map<String,List<String>> lostStorage,
                     List<ConfigProperty> configList)
              throws IOException

The method the worker should execute to register with the block master.

Parameters:

workerId - the worker id of the worker registering

storageTierAliases - a list of storage tier aliases in ordinal order

totalBytesOnTiers - mapping from storage tier alias to total bytes

usedBytesOnTiers - mapping from storage tier alias to used bytes

currentBlocksOnLocation - mapping from storage tier alias to the list of list of blocks

lostStorage - mapping from storage tier alias to the list of lost storage paths

configList - a list of configurations

Throws:

IOException