alluxio.stress

Class CachingBlockMasterClient

java.lang.Object

alluxio.AbstractClient

alluxio.AbstractMasterClient

alluxio.worker.block.BlockMasterClient

alluxio.stress.CachingBlockMasterClient

All Implemented Interfaces:

Client, Closeable, AutoCloseable

public class CachingBlockMasterClient
extends BlockMasterClient

Use this class to avoid the map-list conversion at the client side. This reduces the client side work so the time measurement for the whole RPC better reflects the time taken at the master side. Also reducing the workload on the client side will avoid the test becoming CPU-bound on the client side, when the concurrency is high.

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static class	CachingBlockMasterClient.CachingBlockMapIterator Pre-generate the list so the RPC execution time does not include the conversion time.

Nested classes/interfaces inherited from class alluxio.AbstractClient

AbstractClient.RpcCallable<V>

Field Summary

Fields

Modifier and Type	Field and Description
CachingBlockMasterClient.CachingBlockMapIterator	mBlockBatchIterator

Fields inherited from class alluxio.worker.block.BlockMasterClient

mAsyncClient, mClient

Fields inherited from class alluxio.AbstractClient

mChannel, mClosed, mConnected, mContext, mServerAddress, mServiceVersion, mVersionService

Constructor Summary

Constructors

Constructor and Description
CachingBlockMasterClient(MasterClientContext conf, List<LocationBlockIdListEntry> locationBlockIdList) Creates a new instance and caches the converted proto.
CachingBlockMasterClient(MasterClientContext conf, Map<BlockStoreLocation,List<Long>> blockMap) Creates a new instance and caches the converted proto.

Method Summary

Modifier and Type	Method and Description
List<LocationBlockIdListEntry>	convertBlockListMapToProto(Map<BlockStoreLocation,List<Long>> blockListOnLocation) Converts the block list map to a proto list.
void	registerWithStream(long workerId, List<String> storageTierAliases, Map<String,Long> totalBytesOnTiers, Map<String,Long> usedBytesOnTiers, Map<BlockStoreLocation,List<Long>> currentBlocksOnLocation, Map<String,List<String>> lostStorage, List<ConfigProperty> configList) Registers with the master in a stream.

Methods inherited from class alluxio.worker.block.BlockMasterClient

acquireRegisterLeaseWithBackoff, afterConnect, commitBlock, commitBlockInUfs, getId, getRemoteServiceType, getServiceName, getServiceVersion, heartbeat, notifyWorkerId, register

Methods inherited from class alluxio.AbstractMasterClient

afterDisconnect, beforeConnect, getConfAddress, queryGrpcServerAddress

Methods inherited from class alluxio.AbstractClient

beforeDisconnect, checkVersion, close, connect, connectWithRuntimeException, disconnect, getRemoteHostName, getRemoteServiceVersion, getRemoteSockAddress, isClosed, isConnected, retryRPC, retryRPC

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

mBlockBatchIterator

public CachingBlockMasterClient.CachingBlockMapIterator mBlockBatchIterator

Constructor Detail

CachingBlockMasterClient

public CachingBlockMasterClient(MasterClientContext conf,
                                List<LocationBlockIdListEntry> locationBlockIdList)

Creates a new instance and caches the converted proto.

Parameters:

conf - master client configuration

locationBlockIdList - location block id list

CachingBlockMasterClient

public CachingBlockMasterClient(MasterClientContext conf,
                                Map<BlockStoreLocation,List<Long>> blockMap)

Creates a new instance and caches the converted proto.

Parameters:

conf - master client configuration

blockMap - block lists of each location

Method Detail

convertBlockListMapToProto

public List<LocationBlockIdListEntry> convertBlockListMapToProto(Map<BlockStoreLocation,List<Long>> blockListOnLocation)

Description copied from class: BlockMasterClient

Converts the block list map to a proto list. Because the list is flattened from a map, in the list no two LocationBlockIdListEntry instances shall have the same BlockStoreLocationProto. The uniqueness of BlockStoreLocationProto is determined by tier alias and medium type. That means directories with the same tier alias and medium type will be merged into the same LocationBlockIdListEntry.

Overrides:

convertBlockListMapToProto in class BlockMasterClient

Parameters:

blockListOnLocation - a map from block location to the block list

Returns:

a flattened and deduplicated list

registerWithStream

public void registerWithStream(long workerId,
                               List<String> storageTierAliases,
                               Map<String,Long> totalBytesOnTiers,
                               Map<String,Long> usedBytesOnTiers,
                               Map<BlockStoreLocation,List<Long>> currentBlocksOnLocation,
                               Map<String,List<String>> lostStorage,
                               List<ConfigProperty> configList)
                        throws IOException

Description copied from class: BlockMasterClient

Registers with the master in a stream.

Overrides:

registerWithStream in class BlockMasterClient

Parameters:

workerId - the worker ID

storageTierAliases - storage/tier setup from the configuration

totalBytesOnTiers - the capacity of each tier

usedBytesOnTiers - the current usage of each tier

currentBlocksOnLocation - the blocks in each tier/dir

lostStorage - the lost storage paths

configList - the configuration properties

Throws:

IOException