alluxio.hub.manager.process

Class ManagerProcessContext

java.lang.Object

alluxio.hub.manager.process.ManagerProcessContext

All Implemented Interfaces:

AutoCloseable

@ThreadSafe
public class ManagerProcessContext
extends Object
implements AutoCloseable

This context holds information and methods critical to the function of the Hub Manager. This object may be safely shared between multiple threads.

Field Summary

Fields

Modifier and Type	Field and Description
static org.slf4j.Logger	LOG

Constructor Summary

Constructors

Constructor and Description
ManagerProcessContext(AlluxioConfiguration conf) Creates a new instance of ManagerProcessContext.

Method Summary

Modifier and Type	Method and Description
void	agentHearbeat(AgentHeartbeatRequest req) Register agent with hosted Hub.
void	alluxioClusterHeartbeat(AlluxioCluster cluster) Sends an Alluxio cluster heartbeat to the Hosted Hub.
ApplyMountPointResponse.Payload	applyMount(ApplyMountPointRequest req) Apply mount hdfs command.
void	close()
AlluxioConfigurationSet	configurationSetFor(AlluxioNodeType type) Gets the configuration for a given alluxio node type.
DeleteMountPointResponse.Payload	deleteMount(MountPointInfo mountPointInfo) Remove mount point.
<T> Map<HubNodeAddress,T>	execOnHub(java.util.function.Function<AgentManagerServiceGrpc.AgentManagerServiceBlockingStub,T> action) Execute a function across all Hub agents.
<T> Map<HubNodeAddress,T>	execOnHub(java.util.function.Function<AgentManagerServiceGrpc.AgentManagerServiceBlockingStub,T> action, AlluxioNodeType type) Execute a function across a set of Alluxio nodes on the cluster.
<T> Map<HubNodeAddress,T>	execOnHub(java.util.function.Function<AgentManagerServiceGrpc.AgentManagerServiceBlockingStub,T> action, AlluxioNodeType type, ExecutionType executionType) Execute a function across a set of Alluxio nodes on the cluster.
<T> Map<HubNodeAddress,T>	execOnHub(java.util.function.Function<AgentManagerServiceGrpc.AgentManagerServiceBlockingStub,T> action, String hostname) Execute a function across a set of Alluxio nodes on the cluster.
AlluxioCluster	getAlluxioCluster()
AlluxioConfiguration	getConf()
FileSystem	getFileSystem() Get an instance of the alluxio file system.
HubCluster	getHubCluster()
ListMountPointResponse.Payload	getMountPointList() Get a list of mount points and their information.
GetPrestoConfDirResponse.Payload	getPrestoConf() Get presto conf.
AlluxioConfiguration	getUpdatedProps(AlluxioConfigurationSet configSet) Get updated cluster configuration from the configuration set.
PrestoCatalogListingResult	listCatalogs(String confDir) Gets a list of known UDBs from an Alluxio master.
List<ListFile>	listFiles() List all files.
List<AgentProcessStatusChangeResponse>	processStatusChange(ProcessStatusChangeRequest batchReq) Process state change request for Alluxio nodes.
void	registerAgent(RegisterAgentRequest req) Register agent with hosted Hub.
boolean	removeFile(List<RemoveFile> filesList) Remove a list of files.
SetPrestoConfDirResponse.Payload	setPrestoConfDir(SetPrestoConfDirRequest request) Set Presto configuration directory.
SpeedTestResponse.Payload	speedTest(SpeedTestRequest request) speed test for ufs.
void	startApplyMountPointListener() Starts a request stream observer for HostedManagerServiceGrpc ApplyMountPoint RPC calls.
void	startDeleteMountPointListener() Starts a request stream observer for HostedManagerServiceGrpc DeleteMountPoint RPC calls.
void	startDetectPrestoListener() Starts a request stream observer for HostedManagerServiceGrpc DetectPresto RPC calls.
void	startGetConfigurationSetListener() Starts a request stream observer for HostedManagerServiceGrpc GetConfigurationSet RPC calls.
void	startGetPrestoConfDirListener() Starts a request stream observer for HostedManagerServiceGrpc GetPrestoConfDir RPC calls.
void	startListCatalogListener() Starts a request stream observer for HostedManagerServiceGrpc ListCatalog RPC calls.
void	startListFileListener() Starts a request stream observer for HostedManagerServiceGrpc ListFile RPC calls.
void	startListMountPointsListener() Starts a request stream observer for HostedManagerServiceGrpc ListMountPoints RPC calls.
void	startPingManagerListener() Starts a request stream observer for HostedManagerServiceGrpc PingManager RPC calls.
void	startProcessStatusChangeListener() Starts a request stream observer for HostedManagerServiceGrpc ProcessStatusChange RPC calls.
void	startRemoveFileListener() Starts a request stream observer for HostedManagerServiceGrpc RemoveFile RPC calls.
void	startSetPrestoConfDirListener() Starts a request stream observer for HostedManagerServiceGrpc SetPrestoConfDir RPC calls.
void	startSpeedTestListener() Starts a request stream observer for HostedManagerServiceGrpc SpeedTest RPC calls.
void	startUploadFileListener() Starts a request stream observer for HostedManagerServiceGrpc UploadFile RPC calls.
void	startWriteConfigurationSetListener() Starts a request stream observer for HostedManagerServiceGrpc WriteConfigurationSet RPC calls.
Set<HubNodeAddress>	updateConfigurationFor(AlluxioNodeType type, AlluxioConfigurationSet conf) Updates the configuration for a set of Alluxio nodes.
boolean	uploadFile(List<UploadFile> fileInfoList) Add new files.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

LOG

public static final org.slf4j.Logger LOG

Constructor Detail

ManagerProcessContext

public ManagerProcessContext(AlluxioConfiguration conf)

Creates a new instance of ManagerProcessContext.

Parameters:

conf - alluxio configuration

Method Detail

startPingManagerListener

public void startPingManagerListener()

Starts a request stream observer for HostedManagerServiceGrpc PingManager RPC calls. The PingManagerResponse's HubMetadata must be the PingManagerRequest's HubMetadata in order for the Hosted Hub to properly track where the ping is coming from. This is to handle scenarios where the cluster id changed due to Alluxio journal formats.

startProcessStatusChangeListener

public void startProcessStatusChangeListener()

Starts a request stream observer for HostedManagerServiceGrpc ProcessStatusChange RPC calls.

startGetConfigurationSetListener

public void startGetConfigurationSetListener()

Starts a request stream observer for HostedManagerServiceGrpc GetConfigurationSet RPC calls.

startWriteConfigurationSetListener

public void startWriteConfigurationSetListener()

Starts a request stream observer for HostedManagerServiceGrpc WriteConfigurationSet RPC calls.

startUploadFileListener

public void startUploadFileListener()

Starts a request stream observer for HostedManagerServiceGrpc UploadFile RPC calls.

startListFileListener

public void startListFileListener()

Starts a request stream observer for HostedManagerServiceGrpc ListFile RPC calls.

startRemoveFileListener

public void startRemoveFileListener()

Starts a request stream observer for HostedManagerServiceGrpc RemoveFile RPC calls.

startDetectPrestoListener

public void startDetectPrestoListener()

Starts a request stream observer for HostedManagerServiceGrpc DetectPresto RPC calls.

startListCatalogListener

public void startListCatalogListener()

Starts a request stream observer for HostedManagerServiceGrpc ListCatalog RPC calls.

startSetPrestoConfDirListener

public void startSetPrestoConfDirListener()

Starts a request stream observer for HostedManagerServiceGrpc SetPrestoConfDir RPC calls.

startGetPrestoConfDirListener

public void startGetPrestoConfDirListener()

Starts a request stream observer for HostedManagerServiceGrpc GetPrestoConfDir RPC calls.

startListMountPointsListener

public void startListMountPointsListener()

Starts a request stream observer for HostedManagerServiceGrpc ListMountPoints RPC calls.

startApplyMountPointListener

public void startApplyMountPointListener()

Starts a request stream observer for HostedManagerServiceGrpc ApplyMountPoint RPC calls.

startDeleteMountPointListener

public void startDeleteMountPointListener()

Starts a request stream observer for HostedManagerServiceGrpc DeleteMountPoint RPC calls.

startSpeedTestListener

public void startSpeedTestListener()

Starts a request stream observer for HostedManagerServiceGrpc SpeedTest RPC calls.

getAlluxioCluster

public AlluxioCluster getAlluxioCluster()

Returns:

the AlluxioCluster

getConf

public AlluxioConfiguration getConf()

Returns:

the configuration associated with this context

getHubCluster

public HubCluster getHubCluster()

Returns:

the HubCluster

execOnHub

public <T> Map<HubNodeAddress,T> execOnHub(java.util.function.Function<AgentManagerServiceGrpc.AgentManagerServiceBlockingStub,T> action)

Execute a function across all Hub agents.

Type Parameters:

T - the response type that is returned from the client action

Parameters:

action - a function that supplies a Hub agent client

Returns:

a mapping of responses from each hub agent the action was executed on to the agent's address

execOnHub

public <T> Map<HubNodeAddress,T> execOnHub(java.util.function.Function<AgentManagerServiceGrpc.AgentManagerServiceBlockingStub,T> action,
                                           String hostname)

Execute a function across a set of Alluxio nodes on the cluster.

Type Parameters:

T - the response type that is returned from the client action

Parameters:

action - a function that supplies a Hub agent client

hostname - the host to run on

Returns:

a mapping of responses from each hub agent the action was executed on to the agent's address

execOnHub

public <T> Map<HubNodeAddress,T> execOnHub(java.util.function.Function<AgentManagerServiceGrpc.AgentManagerServiceBlockingStub,T> action,
                                           AlluxioNodeType type)

Execute a function across a set of Alluxio nodes on the cluster.

Type Parameters:

T - the response type that is returned from the client action

Parameters:

action - a function that supplies a Hub agent client

type - the hostname it should run on

Returns:

a mapping of responses from each hub agent the action was executed on to the agent's address

execOnHub

public <T> Map<HubNodeAddress,T> execOnHub(java.util.function.Function<AgentManagerServiceGrpc.AgentManagerServiceBlockingStub,T> action,
                                           AlluxioNodeType type,
                                           ExecutionType executionType)

Execute a function across a set of Alluxio nodes on the cluster.

Type Parameters:

T - the response type that is returned from the client action

Parameters:

action - a function that supplies a Hub agent client

type - the hostname it should run on

executionType - parallel or serial

Returns:

a mapping of responses from each hub agent the action was executed on to the agent's address

processStatusChange

public List<AgentProcessStatusChangeResponse> processStatusChange(ProcessStatusChangeRequest batchReq)

Process state change request for Alluxio nodes.

Parameters:

batchReq - batch state change requests

Returns:

a process status change response

registerAgent

public void registerAgent(RegisterAgentRequest req)

Register agent with hosted Hub.

Parameters:

req - registration request

agentHearbeat

public void agentHearbeat(AgentHeartbeatRequest req)

Register agent with hosted Hub.

Parameters:

req - registration request

alluxioClusterHeartbeat

public void alluxioClusterHeartbeat(AlluxioCluster cluster)

Sends an Alluxio cluster heartbeat to the Hosted Hub. The HubMetadata will always try to use the latest cluster id to handle journal formats. This allows the Hosted Hub to catch cluster id changes. If the response fails, re-initialize the connection with the Hosted Hub with the new cluster id.

Parameters:

cluster - alluxio cluster

configurationSetFor

public AlluxioConfigurationSet configurationSetFor(AlluxioNodeType type)

Gets the configuration for a given alluxio node type.

Parameters:

type - the type of node to get the configuration from

Returns:

a mapping of responses from each hub agent the action was executed on to the agent's address

updateConfigurationFor

public Set<HubNodeAddress> updateConfigurationFor(AlluxioNodeType type,
                                                  AlluxioConfigurationSet conf)

Updates the configuration for a set of Alluxio nodes.

Parameters:

type - the type of node to write the configuration to

conf - the configuration to write

Returns:

a set of the nodes which had their configuration updated

getPrestoConf

public GetPrestoConfDirResponse.Payload getPrestoConf()

Get presto conf.

Returns:

presto conf

getUpdatedProps

public AlluxioConfiguration getUpdatedProps(AlluxioConfigurationSet configSet)

Get updated cluster configuration from the configuration set.

Parameters:

configSet - cluster config set

Returns:

Alluxio configuration

getFileSystem

public FileSystem getFileSystem()

Get an instance of the alluxio file system.

Returns:

alluxio file system or null

close

public void close()
           throws IOException

Specified by:

close in interface AutoCloseable

Throws:

IOException

getMountPointList

public ListMountPointResponse.Payload getMountPointList()

Get a list of mount points and their information.

Returns:

a MountPointList object

applyMount

public ApplyMountPointResponse.Payload applyMount(ApplyMountPointRequest req)

Apply mount hdfs command.

Parameters:

req - validated request

Returns:

response

listCatalogs

public PrestoCatalogListingResult listCatalogs(String confDir)

Gets a list of known UDBs from an Alluxio master.

Parameters:

confDir - the presto configuration directory to search for catalogs

Returns:

the validation results

listFiles

public List<ListFile> listFiles()

List all files.

Returns:

a list of uploaded file info

uploadFile

public boolean uploadFile(List<UploadFile> fileInfoList)

Add new files.

Parameters:

fileInfoList - add a list of file add request

Returns:

true if adds are successful

removeFile

public boolean removeFile(List<RemoveFile> filesList)

Remove a list of files.

Parameters:

filesList - a list of files to be removed

Returns:

true if file removal successful

speedTest

public SpeedTestResponse.Payload speedTest(SpeedTestRequest request)

speed test for ufs.

Parameters:

request - ufs speed test request

Returns:

result of the test

setPrestoConfDir

public SetPrestoConfDirResponse.Payload setPrestoConfDir(SetPrestoConfDirRequest request)

Set Presto configuration directory.

Parameters:

request - presto conf directory request

Returns:

response object

deleteMount

public DeleteMountPointResponse.Payload deleteMount(MountPointInfo mountPointInfo)

Remove mount point.

Parameters:

mountPointInfo - mount point info

Returns:

response