alluxio.hadoop.mapreduce

Class KeyValueInputFormat

java.lang.Object

org.apache.hadoop.mapreduce.InputFormat<org.apache.hadoop.io.BytesWritable,org.apache.hadoop.io.BytesWritable>

alluxio.hadoop.mapreduce.KeyValueInputFormat

@PublicApi
 @ThreadSafe
public final class KeyValueInputFormat
extends org.apache.hadoop.mapreduce.InputFormat<org.apache.hadoop.io.BytesWritable,org.apache.hadoop.io.BytesWritable>

Implementation of org.apache.hadoop.mapred.InputFormat for MapReduce programs to access KeyValueSystem.

It takes a KeyValueSystem URI, and emits key-value pairs stored in the KeyValueStore to org.apache.hadoop.mapred.Mappers.

Constructor Summary

Constructors

Constructor and Description
KeyValueInputFormat() Constructs a new KeyValueInputFormat.

Method Summary

Modifier and Type	Method and Description
org.apache.hadoop.mapreduce.RecordReader<org.apache.hadoop.io.BytesWritable,org.apache.hadoop.io.BytesWritable>	createRecordReader(org.apache.hadoop.mapreduce.InputSplit inputSplit, org.apache.hadoop.mapreduce.TaskAttemptContext taskContext)
List<org.apache.hadoop.mapreduce.InputSplit>	getSplits(org.apache.hadoop.mapreduce.JobContext jobContext) Returns a list of KeyValueInputSplit where each split is one key-value partition.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

KeyValueInputFormat

public KeyValueInputFormat()

Constructs a new KeyValueInputFormat.

Method Detail

getSplits

public List<org.apache.hadoop.mapreduce.InputSplit> getSplits(org.apache.hadoop.mapreduce.JobContext jobContext)
                                                       throws IOException

Returns a list of KeyValueInputSplit where each split is one key-value partition.

Specified by:

getSplits in class org.apache.hadoop.mapreduce.InputFormat<org.apache.hadoop.io.BytesWritable,org.apache.hadoop.io.BytesWritable>

Parameters:

jobContext - MapReduce job configuration

Returns:

list of InputSplits, each split is a partition

Throws:

IOException

createRecordReader

public org.apache.hadoop.mapreduce.RecordReader<org.apache.hadoop.io.BytesWritable,org.apache.hadoop.io.BytesWritable> createRecordReader(org.apache.hadoop.mapreduce.InputSplit inputSplit,
                                                                                                                                          org.apache.hadoop.mapreduce.TaskAttemptContext taskContext)
                                                                                                                                   throws IOException

Specified by:

createRecordReader in class org.apache.hadoop.mapreduce.InputFormat<org.apache.hadoop.io.BytesWritable,org.apache.hadoop.io.BytesWritable>

Throws:

IOException