alluxio.util

Class CommonUtils

java.lang.Object

alluxio.util.CommonUtils

@ThreadSafe
public final class CommonUtils
extends Object

Common utilities shared by all components in Alluxio.

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static class	CommonUtils.ProcessType Alluxio process types.

Field Summary

Fields

Modifier and Type	Field and Description
static AtomicReference<CommonUtils.ProcessType>	PROCESS_TYPE Represents the type of Alluxio process running in this JVM.

Method Summary

Modifier and Type	Method and Description
static <T> String	argsToString(String separator, T... args) Converts varargs of objects to a string.
static IOException	castToIOException(Throwable e) Casts a Throwable to an IOException.
static RuntimeException	closeAndRethrow(com.google.common.io.Closer closer, Throwable t) Closes the Closer and re-throws the Throwable.
static void	closeChannel(io.netty.channel.Channel channel) Closes the netty channel from outside the netty I/O thread.
static void	closeChannelSync(io.netty.channel.Channel channel) Closes the netty channel synchronously.
static String	convertMsToClockTime(long millis) Converts milliseconds to clock time.
static String	convertMsToDate(long millis) Converts a millisecond number to a formatted date String.
static <T> T	createNewClassInstance(Class<T> cls, Class<?>[] ctorClassArgs, Object[] ctorArgs) Creates new instance of a class by calling a constructor that receives ctorClassArgs arguments.
static long	getCurrentMs()
static List<String>	getGroups(String userName) Using CachedGroupMapping to get the group list of a user.
static String	getPrimaryGroupName(String userName) Gets the primary group name of a user.
static Throwable	getRootCause(Throwable e) Gets the root cause of an exception.
static String	getTmpDir()
static List<String>	getUnixGroups(String user) Gets the current user's group list from Unix by running the command 'groups' NOTE.
static String	getValueFromStaticMapping(String mapping, String key) Gets the value with a given key from a static key/value mapping in string format.
static <T> void	invokeAll(List<Callable<T>> callables, long timeout, TimeUnit unit) Executes the given callables, waiting for them to complete (or time out).
static boolean	isLocalHost(WorkerNetAddress address)
static <T> String	listToString(List<T> list) Converts a list of objects to a string.
static <T> java.util.function.Supplier<T>	memoize(java.util.function.Supplier<T> original) Memoize implementation for java.util.function.supplier.
static String	randomAlphaNumString(int length) Generates a random alphanumeric string of the given length.
static byte[]	randomBytes(int length) Generates a random byte array of the given length.
static <T> Iterator<T>	singleElementIterator(T element) Returns an iterator that iterates on a single element.
static void	sleepMs(org.slf4j.Logger logger, long timeMs) Sleeps for the given number of milliseconds, reporting interruptions using the given logger.
static void	sleepMs(long timeMs) Sleeps for the given number of milliseconds.
static String	stripLeadingAndTrailingQuotes(String str) Strips the leading and trailing quotes from the given string.
static String	stripPrefixIfPresent(String key, String prefix) Strips the prefix from the key if it is present.
static String	stripSuffixIfPresent(String key, String suffix) Strips the suffix if it exists.
static String[]	toStringArray(ArrayList<String> src) Parses ArrayList<String> into String[].
static void	unwrapResponse(Protocol.Response response) Unwraps a Protocol.Response.
static void	unwrapResponseFrom(Protocol.Response response, io.netty.channel.Channel channel) Unwraps a Protocol.Response associated with a channel.
static boolean	waitFor(String description, com.google.common.base.Function<Void,Boolean> condition) Waits for a condition to be satisfied.
static boolean	waitFor(String description, com.google.common.base.Function<Void,Boolean> condition, WaitForOptions options) Waits for a condition to be satisfied.
static boolean	waitFor(String description, java.util.function.Supplier<Boolean> condition) Waits for a condition to be satisfied.
static <T> T	waitForResult(String description, com.google.common.base.Function<Void,T> operation, WaitForOptions options) Waits for an operation to return a non-null value with a specified timeout.
static void	warmUpLoop() Common empty loop utility that serves the purpose of warming up the JVM before performance microbenchmarks.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

PROCESS_TYPE

public static final AtomicReference<CommonUtils.ProcessType> PROCESS_TYPE

Represents the type of Alluxio process running in this JVM. NOTE: This will only be set by main methods of Alluxio processes. It will not be set properly for tests. Avoid using this field if at all possible.

Method Detail

getCurrentMs

public static long getCurrentMs()

Returns:

current time in milliseconds

getTmpDir

public static String getTmpDir()

Returns:

a path to a temporary directory based on the user configuration

listToString

public static <T> String listToString(List<T> list)

Converts a list of objects to a string.

Type Parameters:

T - type of the objects

Parameters:

list - list of objects

Returns:

space-separated concatenation of the string representation returned by Object#toString of the individual objects

argsToString

public static <T> String argsToString(String separator,
                                      T... args)

Converts varargs of objects to a string.

Type Parameters:

T - type of the objects

Parameters:

separator - separator string

args - variable arguments

Returns:

concatenation of the string representation returned by Object#toString of the individual objects

toStringArray

public static String[] toStringArray(ArrayList<String> src)

Parses ArrayList<String> into String[].

Parameters:

src - is the ArrayList of strings

Returns:

an array of strings

randomAlphaNumString

public static String randomAlphaNumString(int length)

Generates a random alphanumeric string of the given length.

Parameters:

length - the length

Returns:

a random string

randomBytes

public static byte[] randomBytes(int length)

Generates a random byte array of the given length.

Parameters:

length - the length

Returns:

a random byte array

sleepMs

public static void sleepMs(long timeMs)

Sleeps for the given number of milliseconds.

Parameters:

timeMs - sleep duration in milliseconds

sleepMs

public static void sleepMs(org.slf4j.Logger logger,
                           long timeMs)

Sleeps for the given number of milliseconds, reporting interruptions using the given logger. Unlike Thread.sleep(), this method responds to interrupts by setting the thread interrupt status. This means that callers must check the interrupt status if they need to handle interrupts.

Parameters:

logger - logger for reporting interruptions; no reporting is done if the logger is null

timeMs - sleep duration in milliseconds

warmUpLoop

public static void warmUpLoop()

Common empty loop utility that serves the purpose of warming up the JVM before performance microbenchmarks.

createNewClassInstance

public static <T> T createNewClassInstance(Class<T> cls,
                                           Class<?>[] ctorClassArgs,
                                           Object[] ctorArgs)

Creates new instance of a class by calling a constructor that receives ctorClassArgs arguments.

Type Parameters:

T - type of the object

Parameters:

cls - the class to create

ctorClassArgs - parameters type list of the constructor to initiate, if null default constructor will be called

ctorArgs - the arguments to pass the constructor

Returns:

new class object

Throws:

RuntimeException - if the class cannot be instantiated

getUnixGroups

public static List<String> getUnixGroups(String user)
                                  throws IOException

Gets the current user's group list from Unix by running the command 'groups' NOTE. For non-existing user it will return EMPTY list. This method may return duplicate groups.

Parameters:

user - user name

Returns:

the groups list that the user belongs to. The primary group is returned first

Throws:

IOException

waitFor

public static boolean waitFor(String description,
                              java.util.function.Supplier<Boolean> condition)

Waits for a condition to be satisfied.

Parameters:

description - a description of what causes condition to evaluate to true

condition - the condition to wait on

Returns:

whether the condition returned true; returns false e.g. if the thread was interrupted or hit the timeout with options.isThrowOnTimeout() = false

waitFor

public static boolean waitFor(String description,
                              com.google.common.base.Function<Void,Boolean> condition)

Waits for a condition to be satisfied.

Parameters:

description - a description of what causes condition to evaluate to true

condition - the condition to wait on

Returns:

whether the condition returned true; returns false e.g. if the thread was interrupted or hit the timeout with options.isThrowOnTimeout() = false

waitFor

public static boolean waitFor(String description,
                              com.google.common.base.Function<Void,Boolean> condition,
                              WaitForOptions options)

Waits for a condition to be satisfied.

Parameters:

description - a description of what causes condition to evaluate to true

condition - the condition to wait on

options - the options to use

Returns:

whether the condition returned true; returns false e.g. if the thread was interrupted or hit the timeout with options.isThrowOnTimeout() = false

waitForResult

public static <T> T waitForResult(String description,
                                  com.google.common.base.Function<Void,T> operation,
                                  WaitForOptions options)

Waits for an operation to return a non-null value with a specified timeout.

Type Parameters:

T - the type of the return value

Parameters:

description - the description of this operation

operation - the operation

options - the options to use

Returns:

the return value, null if it times out

getPrimaryGroupName

public static String getPrimaryGroupName(String userName)
                                  throws IOException

Gets the primary group name of a user.

Parameters:

userName - Alluxio user name

Returns:

primary group name

Throws:

IOException

getGroups

public static List<String> getGroups(String userName)
                              throws IOException

Using CachedGroupMapping to get the group list of a user.

Parameters:

userName - Alluxio user name

Returns:

the group list of the user

Throws:

IOException

stripSuffixIfPresent

public static String stripSuffixIfPresent(String key,
                                          String suffix)

Strips the suffix if it exists. This method will leave keys without a suffix unaltered.

Parameters:

key - the key to strip the suffix from

suffix - suffix to remove

Returns:

the key with the suffix removed, or the key unaltered if the suffix is not present

stripPrefixIfPresent

public static String stripPrefixIfPresent(String key,
                                          String prefix)

Strips the prefix from the key if it is present. For example, for input key ufs://my-bucket-name/my-key/file and prefix ufs://my-bucket-name/, the output would be my-key/file. This method will leave keys without a prefix unaltered, ie. my-key/file returns my-key/file.

Parameters:

key - the key to strip

prefix - prefix to remove

Returns:

the key without the prefix

stripLeadingAndTrailingQuotes

public static String stripLeadingAndTrailingQuotes(String str)

Strips the leading and trailing quotes from the given string. E.g. return 'alluxio' for input '"alluxio"'.

Parameters:

str - The string to strip

Returns:

The string without the leading and trailing quotes

getValueFromStaticMapping

public static String getValueFromStaticMapping(String mapping,
                                               String key)

Gets the value with a given key from a static key/value mapping in string format. E.g. with mapping "id1=user1;id2=user2", it returns "user1" with key "id1". It returns null if the given key does not exist in the mapping.

Parameters:

mapping - the "key=value" mapping in string format separated by ";"

key - the key to query

Returns:

the mapped value if the key exists, otherwise returns ""

getRootCause

public static Throwable getRootCause(Throwable e)

Gets the root cause of an exception.

Parameters:

e - the exception

Returns:

the root cause

castToIOException

public static IOException castToIOException(Throwable e)

Casts a Throwable to an IOException.

Parameters:

e - the throwable

Returns:

the IO exception

singleElementIterator

public static <T> Iterator<T> singleElementIterator(T element)

Returns an iterator that iterates on a single element.

Type Parameters:

T - the type of the element

Parameters:

element - the element

Returns:

the iterator

invokeAll

public static <T> void invokeAll(List<Callable<T>> callables,
                                 long timeout,
                                 TimeUnit unit)
                          throws TimeoutException,
                                 Exception

Executes the given callables, waiting for them to complete (or time out). If a callable throws an exception, that exception will be re-thrown from this method.

Type Parameters:

T - the return type of the callables

Parameters:

callables - the callables to execute

timeout - the maximum time to wait

unit - the time unit of the timeout argument

Throws:

Exception - if any of the callables throws an exception

TimeoutException

closeAndRethrow

public static RuntimeException closeAndRethrow(com.google.common.io.Closer closer,
                                               Throwable t)
                                        throws IOException

Closes the Closer and re-throws the Throwable. Any exceptions thrown while closing the Closer will be added as suppressed exceptions to the Throwable. This method always throws the given Throwable, wrapping it in a RuntimeException if it's a non-IOException checked exception. Note: This method will wrap non-IOExceptions in RuntimeException. Do not use this method in methods which throw non-IOExceptions.

 Closer closer = new Closer();
 try {
   Closeable c = closer.register(new Closeable());
 } catch (Throwable t) {
   throw closeAndRethrow(closer, t);
 }
 

Parameters:

closer - the Closer to close

t - the Throwable to re-throw

Returns:

this method never returns

Throws:

IOException

unwrapResponse

public static void unwrapResponse(Protocol.Response response)
                           throws AlluxioStatusException

Unwraps a Protocol.Response.

Parameters:

response - the response

Throws:

AlluxioStatusException

unwrapResponseFrom

public static void unwrapResponseFrom(Protocol.Response response,
                                      io.netty.channel.Channel channel)
                               throws AlluxioStatusException

Unwraps a Protocol.Response associated with a channel.

Parameters:

response - the response

channel - the channel that receives this response

Throws:

AlluxioStatusException

isLocalHost

public static boolean isLocalHost(WorkerNetAddress address)

Parameters:

address - the Alluxio worker network address

Returns:

true if the worker is local

closeChannel

public static void closeChannel(io.netty.channel.Channel channel)

Closes the netty channel from outside the netty I/O thread. NOTE: Be careful when holding any lock that can be acquired in the netty I/O thread when calling this function to avoid having deadlocks.

Parameters:

channel - the netty channel

closeChannelSync

public static void closeChannelSync(io.netty.channel.Channel channel)

Closes the netty channel synchronously. Usually do not do this since this can take long time if the server is not responsive.

Parameters:

channel - the netty channel

convertMsToDate

public static String convertMsToDate(long millis)

Converts a millisecond number to a formatted date String.

Parameters:

millis - a long millisecond number

Returns:

formatted date String

convertMsToClockTime

public static String convertMsToClockTime(long millis)

Converts milliseconds to clock time.

Parameters:

millis - milliseconds

Returns:

input encoded as clock time

memoize

public static <T> java.util.function.Supplier<T> memoize(java.util.function.Supplier<T> original)

Memoize implementation for java.util.function.supplier.

Type Parameters:

T - the object type

Parameters:

original - the original supplier

Returns:

the supplier with memorization