alluxio.master.scheduler

Class Scheduler

java.lang.Object

alluxio.master.scheduler.Scheduler

@ThreadSafe
public final class Scheduler
extends Object

The Scheduler which controls jobs. It is responsible for managing active workers, updating jobs and update job information to job meta store. The workflow is: 1. Submit a job to the scheduler. 2. The scheduler will pull the task from the job and assign the task to a worker. 3. The worker will execute the task and report the result to the job. 4. The job will update the progress. And schedule the next task if the job is not done. 5. One worker would have one task running for one job description at a time.

Constructor Summary

Constructors

Constructor and Description
Scheduler(WorkerProvider workerProvider, JobMetaStore jobMetaStore) Constructor.

Method Summary

Modifier and Type	Method and Description
void	cleanupStaleJob() Removes all finished jobs outside the retention time.
Map<WorkerInfo,CloseableResource<BlockWorkerClient>>	getActiveWorkers() Get active workers.
String	getJobProgress(JobDescription jobDescription, JobProgressReportFormat format, boolean verbose) Get the job's progress report.
Map<JobDescription,Job<?>>	getJobs() Get jobs.
void	start() Start scheduler.
void	stop() Stop scheduler.
boolean	stopJob(JobDescription jobDescription) Stop a job.
boolean	submitJob(Job<?> job) Submit a job.
void	updateWorkers() Refresh active workers.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

Scheduler

public Scheduler(WorkerProvider workerProvider,
                 JobMetaStore jobMetaStore)

Constructor.

Parameters:

workerProvider - workerProvider

jobMetaStore - jobMetaStore

Method Detail

start

public void start()

Start scheduler.

stop

public void stop()

Stop scheduler.

submitJob

public boolean submitJob(Job<?> job)

Submit a job.

Parameters:

job - the job

Returns:

true if the job is new, false if the job has already been submitted

Throws:

ResourceExhaustedRuntimeException - if the job cannot be submitted because the scheduler is at capacity

UnavailableRuntimeException - if the job cannot be submitted because the meta store is not ready

stopJob

public boolean stopJob(JobDescription jobDescription)

Stop a job.

Parameters:

jobDescription - job identifier

Returns:

true if the job is stopped, false if the job does not exist or has already finished

getJobProgress

public String getJobProgress(JobDescription jobDescription,
                             JobProgressReportFormat format,
                             boolean verbose)

Get the job's progress report.

Parameters:

jobDescription - job identifier

format - progress report format

verbose - whether to include details on failed files and failures

Returns:

the progress report

Throws:

NotFoundRuntimeException - if the job does not exist

AlluxioRuntimeException - if any other Alluxio exception occurs

getActiveWorkers

public Map<WorkerInfo,CloseableResource<BlockWorkerClient>> getActiveWorkers()

Get active workers.

Returns:

active workers

cleanupStaleJob

public void cleanupStaleJob()

Removes all finished jobs outside the retention time.

updateWorkers

public void updateWorkers()

Refresh active workers.

getJobs

public Map<JobDescription,Job<?>> getJobs()

Get jobs.

Returns:

jobs