alluxio.hadoop

Class HadoopUtils

java.lang.Object

alluxio.hadoop.HadoopUtils

@ThreadSafe
public final class HadoopUtils
extends Object

Utility class for using Alluxio with Hadoop.

Method Summary

Modifier and Type	Method and Description
static void	addSwiftCredentials(org.apache.hadoop.conf.Configuration configuration) Adds Swift keys to the given Hadoop Configuration object if the user has specified them using System properties, and they're not already set.
static String	getAlluxioFileName(String path) Given a String path, returns an equivalent Alluxio path.
static String	getPathWithoutScheme(org.apache.hadoop.fs.Path path) Given a Path path, it returns the path component of its URI, which has the form scheme://authority/path.
static AlluxioConfiguration	toAlluxioConf(org.apache.hadoop.conf.Configuration conf)
static URIStatus	toAlluxioUriStatus(org.apache.hadoop.fs.FileStatus status)
static org.apache.hadoop.fs.Path	toPath(AlluxioURI alluxioURI)
static String	toStringHadoopFileSplit(org.apache.hadoop.mapred.FileSplit fs) Returns a string representation of a Hadoop FileSplit.
static String	toStringHadoopFileStatus(org.apache.hadoop.fs.FileStatus fs) Returns a string representation of a Hadoop FileStatus.
static String	toStringHadoopInputSplit(org.apache.hadoop.mapred.InputSplit is) Returns a string representation of a InputSplit.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Method Detail

getPathWithoutScheme

public static String getPathWithoutScheme(org.apache.hadoop.fs.Path path)

Given a Path path, it returns the path component of its URI, which has the form scheme://authority/path.

Parameters:

path - an HDFS Path

Returns:

the path component of the Path URI

getAlluxioFileName

public static String getAlluxioFileName(String path)

Given a String path, returns an equivalent Alluxio path.

Parameters:

path - the path to parse

Returns:

a valid Alluxio path

toStringHadoopFileSplit

public static String toStringHadoopFileSplit(org.apache.hadoop.mapred.FileSplit fs)

Returns a string representation of a Hadoop FileSplit.

Parameters:

fs - Hadoop FileSplit

Returns:

its string representation

toStringHadoopFileStatus

public static String toStringHadoopFileStatus(org.apache.hadoop.fs.FileStatus fs)

Returns a string representation of a Hadoop FileStatus.

Parameters:

fs - Hadoop FileStatus

Returns:

its string representation

toStringHadoopInputSplit

public static String toStringHadoopInputSplit(org.apache.hadoop.mapred.InputSplit is)

Returns a string representation of a InputSplit.

Parameters:

is - Hadoop InputSplit

Returns:

its string representation

addSwiftCredentials

public static void addSwiftCredentials(org.apache.hadoop.conf.Configuration configuration)

Adds Swift keys to the given Hadoop Configuration object if the user has specified them using System properties, and they're not already set. This function is duplicated from alluxio.underfs.hdfs.HdfsUnderFileSystemUtils, to prevent the module alluxio-core-client-hdfs from depending on the module alluxio-underfs.

Parameters:

configuration - Hadoop configuration

toPath

public static org.apache.hadoop.fs.Path toPath(AlluxioURI alluxioURI)

Parameters:

alluxioURI - Alluxio uri

Returns:

corresponding Hadoop Path instance

toAlluxioConf

public static AlluxioConfiguration toAlluxioConf(org.apache.hadoop.conf.Configuration conf)

Parameters:

conf - Hadoop conf

Returns:

Alluxio configuration merged from Hadoop conf

toAlluxioUriStatus

public static URIStatus toAlluxioUriStatus(org.apache.hadoop.fs.FileStatus status)

Parameters:

status - Hadoop file status

Returns:

corresponding Alluxio uri status instance