alluxio.job

Interface JobDefinition<T extends JobConfig,P extends Serializable,R extends Serializable>

Type Parameters:

T - the job configuration

P - the parameters to pass to each task

R - the return type from the task

All Known Implementing Classes:

AbstractVoidJobDefinition, CompactDefinition, CompositeDefinition, EvictDefinition, LoadDefinition, MigrateDefinition, MoveDefinition, PersistDefinition, ReplicateDefinition

public interface JobDefinition<T extends JobConfig,P extends Serializable,R extends Serializable>

A job definition. A definition has two important parts: (1) a selectExecutors(JobConfig, List, SelectExecutorsContext) method runs at the master node and selects the workers to run the executors. (2) a runTask(JobConfig, Serializable, RunTaskContext) method runs at each selected executor on the worker node.

Method Summary

Modifier and Type	Method and Description
Class<T>	getJobConfigClass()
String	join(T config, Map<WorkerInfo,R> taskResults) Joins the task results on the master.
R	runTask(T config, P args, RunTaskContext runTaskContext) Runs the task in the executor.
Map<WorkerInfo,P>	selectExecutors(T config, List<WorkerInfo> jobWorkerInfoList, SelectExecutorsContext selectExecutorsContext) Selects the workers to run the task.

Method Detail

getJobConfigClass

Class<T> getJobConfigClass()

Returns:

the class of the associated JobConfig

selectExecutors

Map<WorkerInfo,P> selectExecutors(T config,
                                  List<WorkerInfo> jobWorkerInfoList,
                                  SelectExecutorsContext selectExecutorsContext)
                           throws Exception

Selects the workers to run the task.

Parameters:

config - the job configuration

jobWorkerInfoList - the list of available workers' information

selectExecutorsContext - the context containing information used to select executors

Returns:

a map of selected workers to the parameters to pass along

Throws:

Exception - if any error occurs

runTask

R runTask(T config,
          P args,
          RunTaskContext runTaskContext)
   throws Exception

Runs the task in the executor.

Parameters:

config - the job configuration

args - the arguments passed in

runTaskContext - the context containing information used to execute a task

Returns:

the task result

Throws:

Exception - if any error occurs

join

String join(T config,
            Map<WorkerInfo,R> taskResults)
     throws Exception

Joins the task results on the master.

Parameters:

config - the job configuration

taskResults - the task results

Returns:

the joined results

Throws:

Exception - if any error occurs