CommonUtils (Alluxio Parent 2.6.2 API)

java.lang.Object
- alluxio.util.CommonUtils

@ThreadSafe
public final class CommonUtils
extends Object

Common utilities shared by all components in Alluxio.

Nested Class Summary

Nested Classes
Modifier and Type Class and Description

static class CommonUtils.ProcessType
Alluxio process types.

Nested Classes
Modifier and Type	Class and Description
`static class`	`CommonUtils.ProcessType` Alluxio process types.

Field Summary

Fields
Modifier and Type Field and Description

static AtomicReference<CommonUtils.ProcessType> PROCESS_TYPE
Represents the type of Alluxio process running in this JVM.

Fields
Modifier and Type	Field and Description
`static AtomicReference<CommonUtils.ProcessType>`	`PROCESS_TYPE` Represents the type of Alluxio process running in this JVM.

Method Summary

All Methods Static Methods Concrete Methods
Modifier and Type	Method and Description
`static <T> String`	`argsToString(String separator, T... args)` Converts varargs of objects to a string.
`static IOException`	`castToIOException(Throwable e)` Casts a `Throwable` to an `IOException`.
`static RuntimeException`	`closeAndRethrow(com.google.common.io.Closer closer, Throwable t)` Closes the Closer and re-throws the Throwable.
`static RuntimeException`	`closeAndRethrowRuntimeException(com.google.common.io.Closer closer, Throwable t)` Similar to `closeAndRethrow(com.google.common.io.Closer, java.lang.Throwable)` but always return a RuntimeException.
`static Map<String,byte[]>`	`convertFromByteString(Map<String,com.google.protobuf.ByteString> input)`
`static String`	`convertMsToClockTime(long millis)` Converts milliseconds to clock time.
`static String`	`convertMsToDate(long millis, String dateFormatPattern)` Converts a millisecond number to a formatted date String.
`static Map<String,com.google.protobuf.ByteString>`	`convertToByteString(Map<String,byte[]> input)`
`static <T> T`	`createNewClassInstance(Class<T> cls, Class<?>[] ctorClassArgs, Object[] ctorArgs)` Creates new instance of a class by calling a constructor that receives ctorClassArgs arguments.
`static Thread`	`createProgressThread(long intervalMs, PrintStream stream)` Creates a thread which will write "." to the given print stream at the given interval.
`static Thread`	`createProgressThread(PrintStream stream)` Convenience method for calling `createProgressThread(long, PrintStream)` with an interval of 2 seconds.
`static long`	`getCurrentMs()`
`static List<String>`	`getGroups(String userName, AlluxioConfiguration conf)` Using `CachedGroupMapping` to get the group list of a user.
`static int`	`getJavaVersion()`
`static String`	`getPrimaryGroupName(String userName, AlluxioConfiguration conf)` Gets the primary group name of a user.
`static Throwable`	`getRootCause(Throwable e)` Gets the root cause of an exception.
`static String`	`getTmpDir(List<String> tmpDirs)`
`static List<String>`	`getUnixGroups(String user)` Gets the current user's group list from Unix by running the command 'groups' NOTE.
`static String`	`getValueFromStaticMapping(String mapping, String key)` Gets the value with a given key from a static key/value mapping in string format.
`static String`	`getWorkerDataDirectory(String storageDir, AlluxioConfiguration conf)`
`static <T> void`	`invokeAll(ExecutorService service, List<Callable<T>> callables, long timeoutMs)` Executes the given callables, waiting for them to complete (or time out).
`static <T> void`	`invokeAll(List<Callable<T>> callables, long timeoutMs)` Executes the given callables, waiting for them to complete (or time out).
`static boolean`	`isAddressReachable(String hostname, int port, int timeoutMs)` Validates whether a network address is reachable.
`static boolean`	`isLocalHost(WorkerNetAddress address, AlluxioConfiguration conf)`
`static <T> String`	`listToString(List<T> list)` Converts a list of objects to a string.
`static <T> java.util.function.Supplier<T>`	`memoize(java.util.function.Supplier<T> original)` Memoize implementation for java.util.function.supplier.
`static int`	`parseMajorVersion(String version)`
`static <T> List<List<T>>`	`partition(List<T> list, int numLists)` Partitions a list into numLists many lists each with around list.size() / numLists elements.
`static String`	`randomAlphaNumString(int length)` Generates a random alphanumeric string of the given length.
`static byte[]`	`randomBytes(int length)` Generates a random byte array of the given length.
`static List<File>`	`recursiveListLocalDir(File dir)` Recursively lists a local dir and all its subdirs and return all the files.
`static <T> Iterator<T>`	`singleElementIterator(T element)` Returns an iterator that iterates on a single element.
`static void`	`sleepMs(org.slf4j.Logger logger, long timeMs)` Sleeps for the given number of milliseconds, reporting interruptions using the given logger.
`static void`	`sleepMs(long timeMs)` Sleeps for the given number of milliseconds.
`static String`	`stripLeadingAndTrailingQuotes(String str)` Strips the leading and trailing quotes from the given string.
`static String`	`stripPrefixIfPresent(String key, String prefix)` Strips the prefix from the key if it is present.
`static String`	`stripSuffixIfPresent(String key, String suffix)` Strips the suffix if it exists.
`static String[]`	`toStringArray(ArrayList<String> src)` Parses `ArrayList<String>` into `String[]`.
`static void`	`unwrapResponse(Protocol.Response response)` Unwraps a `Protocol.Response`.
`static void`	`unwrapResponseFrom(Protocol.Response response, io.netty.channel.Channel channel)` Unwraps a `Protocol.Response` associated with a channel.
`static void`	`waitFor(String description, java.util.function.Supplier<Boolean> condition)` Waits for a condition to be satisfied.
`static void`	`waitFor(String description, java.util.function.Supplier<Boolean> condition, WaitForOptions options)` Waits for a condition to be satisfied.
`static <T> T`	`waitForResult(String description, java.util.function.Supplier<T> objectSupplier, java.util.function.Function<T,Boolean> condition)` Waits for the object to meet a certain condition.
`static <T> T`	`waitForResult(String description, java.util.function.Supplier<T> objectSupplier, java.util.function.Function<T,Boolean> condition, WaitForOptions options)` Waits for the object to meet a certain condition.
`static void`	`warmUpLoop()` Common empty loop utility that serves the purpose of warming up the JVM before performance microbenchmarks.

Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

- Field Detail
  - PROCESS_TYPE
```
public static final AtomicReference<CommonUtils.ProcessType> PROCESS_TYPE
```
    Represents the type of Alluxio process running in this JVM. NOTE: This will only be set by main methods of Alluxio processes. It will not be set properly for tests. Avoid using this field if at all possible.
- Method Detail
  - createProgressThread
```
public static Thread createProgressThread(PrintStream stream)
```
    Convenience method for calling createProgressThread(long, PrintStream) with an interval of 2 seconds.
    
    Parameters:
    
    stream - the print stream to write to
    
    Returns:
    
    the thread
  - createProgressThread
```
public static Thread createProgressThread(long intervalMs,
                                          PrintStream stream)
```
    Creates a thread which will write "." to the given print stream at the given interval. The created thread is not started by this method. The created thread will be a daemon thread and will halt when interrupted.
    
    Parameters:
    
    intervalMs - the time interval in milliseconds between writes
    
    stream - the print stream to write to
    
    Returns:
    
    the thread
  - getCurrentMs
```
public static long getCurrentMs()
```
    Returns:
    
    current time in milliseconds
  - getJavaVersion
```
public static int getJavaVersion()
```
    Returns:
    
    the current jvm major version, 8 for java 1.8 or 11 for java 11
  - getTmpDir
```
public static String getTmpDir(List<String> tmpDirs)
```
    Parameters:
    
    tmpDirs - the list of possible temporary directories to pick from
    
    Returns:
    
    a path to a temporary directory based on the user configuration
  - getWorkerDataDirectory
```
public static String getWorkerDataDirectory(String storageDir,
                                            AlluxioConfiguration conf)
```
    Parameters:
    
    storageDir - the root of a storage directory in tiered storage
    
    conf - Alluxio's current configuration
    
    Returns:
    
    the worker data folder path after each storage directory, the final path will be like "/mnt/ramdisk/alluxioworker" for storage dir "/mnt/ramdisk" by appending {@link PropertyKey#WORKER_DATA_FOLDER).
  - listToString
```
public static <T> String listToString(List<T> list)
```
    Converts a list of objects to a string.
    
    Type Parameters:
    
    T - type of the objects
    
    Parameters:
    
    list - list of objects
    
    Returns:
    
    space-separated concatenation of the string representation returned by Object#toString of the individual objects
  - argsToString
```
public static <T> String argsToString(String separator,
                                      T... args)
```
    Converts varargs of objects to a string.
    
    Type Parameters:
    
    T - type of the objects
    
    Parameters:
    
    separator - separator string
    
    args - variable arguments
    
    Returns:
    
    concatenation of the string representation returned by Object#toString of the individual objects
  - toStringArray
```
public static String[] toStringArray(ArrayList<String> src)
```
    Parses ArrayList<String> into String[].
    
    Parameters:
    
    src - is the ArrayList of strings
    
    Returns:
    
    an array of strings
  - randomAlphaNumString
```
public static String randomAlphaNumString(int length)
```
    Generates a random alphanumeric string of the given length.
    
    Parameters:
    
    length - the length
    
    Returns:
    
    a random string
  - randomBytes
```
public static byte[] randomBytes(int length)
```
    Generates a random byte array of the given length.
    
    Parameters:
    
    length - the length
    
    Returns:
    
    a random byte array
  - sleepMs
```
public static void sleepMs(long timeMs)
```
    Sleeps for the given number of milliseconds.
    
    Parameters:
    
    timeMs - sleep duration in milliseconds
  - sleepMs
```
public static void sleepMs(org.slf4j.Logger logger,
                           long timeMs)
```
    Sleeps for the given number of milliseconds, reporting interruptions using the given logger. Unlike Thread.sleep(), this method responds to interrupts by setting the thread interrupt status. This means that callers must check the interrupt status if they need to handle interrupts.
    
    Parameters:
    
    logger - logger for reporting interruptions; no reporting is done if the logger is null
    
    timeMs - sleep duration in milliseconds
  - warmUpLoop
```
public static void warmUpLoop()
```
    Common empty loop utility that serves the purpose of warming up the JVM before performance microbenchmarks.
  - createNewClassInstance
```
public static <T> T createNewClassInstance(Class<T> cls,
                                           Class<?>[] ctorClassArgs,
                                           Object[] ctorArgs)
```
    Creates new instance of a class by calling a constructor that receives ctorClassArgs arguments.
    
    Type Parameters:
    
    T - type of the object
    
    Parameters:
    
    cls - the class to create
    
    ctorClassArgs - parameters type list of the constructor to initiate, if null default constructor will be called
    
    ctorArgs - the arguments to pass the constructor
    
    Returns:
    
    new class object
    
    Throws:
    
    RuntimeException - if the class cannot be instantiated
  - getUnixGroups
```
public static List<String> getUnixGroups(String user)
                                  throws IOException
```
    Gets the current user's group list from Unix by running the command 'groups' NOTE. For non-existing user it will return EMPTY list. This method may return duplicate groups.
    
    Parameters:
    
    user - user name
    
    Returns:
    
    the groups list that the user belongs to. The primary group is returned first
    
    Throws:
    
    IOException
  - waitFor
```
public static void waitFor(String description,
                           java.util.function.Supplier<Boolean> condition)
                    throws InterruptedException,
                           TimeoutException
```
    Waits for a condition to be satisfied.
    
    Parameters:
    
    description - a description of what causes condition to evaluate to true
    
    condition - the condition to wait on
    
    Throws:
    
    TimeoutException - if the function times out while waiting for the condition to be true
    
    InterruptedException
  - waitFor
```
public static void waitFor(String description,
                           java.util.function.Supplier<Boolean> condition,
                           WaitForOptions options)
                    throws InterruptedException,
                           TimeoutException
```
    Waits for a condition to be satisfied.
    
    Parameters:
    
    description - a description of what causes condition to evaluate to true
    
    condition - the condition to wait on
    
    options - the options to use
    
    Throws:
    
    TimeoutException - if the function times out while waiting for the condition to be true
    
    InterruptedException
  - waitForResult
```
public static <T> T waitForResult(String description,
                                  java.util.function.Supplier<T> objectSupplier,
                                  java.util.function.Function<T,Boolean> condition)
                           throws TimeoutException,
                                  InterruptedException
```
    Waits for the object to meet a certain condition.
    
    Type Parameters:
    
    T - type of the object
    
    Parameters:
    
    description - a description of what causes condition to be met
    
    objectSupplier - the object to check the condition for
    
    condition - the condition to wait on
    
    Returns:
    
    the object
    
    Throws:
    
    TimeoutException - if the function times out while waiting for the condition to be true
    
    InterruptedException - if the thread was interrupted
  - waitForResult
```
public static <T> T waitForResult(String description,
                                  java.util.function.Supplier<T> objectSupplier,
                                  java.util.function.Function<T,Boolean> condition,
                                  WaitForOptions options)
                           throws TimeoutException,
                                  InterruptedException
```
    Waits for the object to meet a certain condition.
    
    Type Parameters:
    
    T - type of the object
    
    Parameters:
    
    description - a description of what causes condition to be met
    
    objectSupplier - the object to check the condition for
    
    condition - the condition to wait on
    
    options - the options to use
    
    Returns:
    
    the object
    
    Throws:
    
    TimeoutException - if the function times out while waiting for the condition to be true
    
    InterruptedException - if the thread was interrupted
  - getPrimaryGroupName
```
public static String getPrimaryGroupName(String userName,
                                         AlluxioConfiguration conf)
                                  throws IOException
```
    Gets the primary group name of a user.
    
    Parameters:
    
    userName - Alluxio user name
    
    conf - Alluxio configuration
    
    Returns:
    
    primary group name
    
    Throws:
    
    IOException
  - getGroups
```
public static List<String> getGroups(String userName,
                                     AlluxioConfiguration conf)
                              throws IOException
```
    Using CachedGroupMapping to get the group list of a user.
    
    Parameters:
    
    userName - Alluxio user name
    
    conf - Alluxio configuration
    
    Returns:
    
    the group list of the user
    
    Throws:
    
    IOException
  - stripSuffixIfPresent
```
public static String stripSuffixIfPresent(String key,
                                          String suffix)
```
    Strips the suffix if it exists. This method will leave keys without a suffix unaltered.
    
    Parameters:
    
    key - the key to strip the suffix from
    
    suffix - suffix to remove
    
    Returns:
    
    the key with the suffix removed, or the key unaltered if the suffix is not present
  - stripPrefixIfPresent
```
public static String stripPrefixIfPresent(String key,
                                          String prefix)
```
    Strips the prefix from the key if it is present. For example, for input key ufs://my-bucket-name/my-key/file and prefix ufs://my-bucket-name/, the output would be my-key/file. This method will leave keys without a prefix unaltered, ie. my-key/file returns my-key/file.
    
    Parameters:
    
    key - the key to strip
    
    prefix - prefix to remove
    
    Returns:
    
    the key without the prefix
  - stripLeadingAndTrailingQuotes
```
public static String stripLeadingAndTrailingQuotes(String str)
```
    Strips the leading and trailing quotes from the given string. E.g. return 'alluxio' for input '"alluxio"'.
    
    Parameters:
    
    str - The string to strip
    
    Returns:
    
    The string without the leading and trailing quotes
  - getValueFromStaticMapping
```
@Nullable
public static String getValueFromStaticMapping(String mapping,
                                                         String key)
```
    Gets the value with a given key from a static key/value mapping in string format. E.g. with mapping "id1=user1;id2=user2", it returns "user1" with key "id1". It returns null if the given key does not exist in the mapping.
    
    Parameters:
    
    mapping - the "key=value" mapping in string format separated by ";"
    
    key - the key to query
    
    Returns:
    
    the mapped value if the key exists, otherwise returns null
  - getRootCause
```
public static Throwable getRootCause(Throwable e)
```
    Gets the root cause of an exception. It stops at encountering gRPC's StatusRuntimeException.
    
    Parameters:
    
    e - the exception
    
    Returns:
    
    the root cause
  - castToIOException
```
public static IOException castToIOException(Throwable e)
```
    Casts a Throwable to an IOException.
    
    Parameters:
    
    e - the throwable
    
    Returns:
    
    the IO exception
  - singleElementIterator
```
public static <T> Iterator<T> singleElementIterator(T element)
```
    Returns an iterator that iterates on a single element.
    
    Type Parameters:
    
    T - the type of the element
    
    Parameters:
    
    element - the element
    
    Returns:
    
    the iterator
  - invokeAll
```
public static <T> void invokeAll(List<Callable<T>> callables,
                                 long timeoutMs)
                          throws TimeoutException,
                                 ExecutionException
```
    Executes the given callables, waiting for them to complete (or time out). If a callable throws an exception, that exception will be re-thrown from this method.
    
    Type Parameters:
    
    T - the return type of the callables
    
    Parameters:
    
    callables - the callables to execute
    
    timeoutMs - time to wait for the callables to complete, in milliseconds
    
    Throws:
    
    TimeoutException - if the callables don't complete before the timeout
    
    ExecutionException - if any of the callables throws an exception
  - invokeAll
```
public static <T> void invokeAll(ExecutorService service,
                                 List<Callable<T>> callables,
                                 long timeoutMs)
                          throws TimeoutException,
                                 ExecutionException
```
    Executes the given callables, waiting for them to complete (or time out). If a callable throws an exception, that exception will be re-thrown from this method. If the current thread is interrupted, an Exception will be thrown.
    
    Type Parameters:
    
    T - the return type of the callables
    
    Parameters:
    
    service - the service to execute the callables
    
    callables - the callables to execute
    
    timeoutMs - time to wait for the callables to complete, in milliseconds
    
    Throws:
    
    TimeoutException - if the callables don't complete before the timeout
    
    ExecutionException - if any of the callables throws an exception
  - closeAndRethrow
```
public static RuntimeException closeAndRethrow(com.google.common.io.Closer closer,
                                               Throwable t)
                                        throws IOException
```
    Closes the Closer and re-throws the Throwable. Any exceptions thrown while closing the Closer will be added as suppressed exceptions to the Throwable. This method always throws the given Throwable, wrapping it in a RuntimeException if it's a non-IOException checked exception. Note: This method will wrap non-IOExceptions in RuntimeException. Do not use this method in methods which throw non-IOExceptions.
```
 Closer closer = new Closer();
 try {
   Closeable c = closer.register(new Closeable());
 } catch (Throwable t) {
   throw closeAndRethrow(closer, t);
 }
 
```
    Parameters:
    
    closer - the Closer to close
    
    t - the Throwable to re-throw
    
    Returns:
    
    this method never returns
    
    Throws:
    
    IOException
  - closeAndRethrowRuntimeException
```
public static RuntimeException closeAndRethrowRuntimeException(com.google.common.io.Closer closer,
                                                               Throwable t)
```
    Similar to closeAndRethrow(com.google.common.io.Closer, java.lang.Throwable) but always return a RuntimeException.
    
    Parameters:
    
    closer - the Closer to close
    
    t - the Throwable to re-throw
    
    Returns:
    
    this method never returns
  - unwrapResponse
```
public static void unwrapResponse(Protocol.Response response)
                           throws AlluxioStatusException
```
    Unwraps a Protocol.Response.
    
    Parameters:
    
    response - the response
    
    Throws:
    
    AlluxioStatusException
  - unwrapResponseFrom
```
public static void unwrapResponseFrom(Protocol.Response response,
                                      io.netty.channel.Channel channel)
                               throws AlluxioStatusException
```
    Unwraps a Protocol.Response associated with a channel.
    
    Parameters:
    
    response - the response
    
    channel - the channel that receives this response
    
    Throws:
    
    AlluxioStatusException
  - isLocalHost
```
public static boolean isLocalHost(WorkerNetAddress address,
                                  AlluxioConfiguration conf)
```
    Parameters:
    
    address - the Alluxio worker network address
    
    conf - Alluxio configuration
    
    Returns:
    
    true if the worker is local
  - convertMsToDate
```
public static String convertMsToDate(long millis,
                                     String dateFormatPattern)
```
    Converts a millisecond number to a formatted date String.
    
    Parameters:
    
    millis - a long millisecond number
    
    dateFormatPattern - the date format to follow when converting. i.e. mm-dd-yyyy
    
    Returns:
    
    formatted date String
  - convertMsToClockTime
```
public static String convertMsToClockTime(long millis)
```
    Converts milliseconds to clock time.
    
    Parameters:
    
    millis - milliseconds
    
    Returns:
    
    input encoded as clock time
  - convertToByteString
```
public static Map<String,com.google.protobuf.ByteString> convertToByteString(Map<String,byte[]> input)
```
    Parameters:
    
    input - the input map
    
    Returns:
    
    a map using protobuf ByteString for values instead of byte[]
  - convertFromByteString
```
public static Map<String,byte[]> convertFromByteString(Map<String,com.google.protobuf.ByteString> input)
```
    Parameters:
    
    input - the input map
    
    Returns:
    
    a map using byte[] for values instead of protobuf ByteString
  - memoize
```
public static <T> java.util.function.Supplier<T> memoize(java.util.function.Supplier<T> original)
```
    Memoize implementation for java.util.function.supplier.
    
    Type Parameters:
    
    T - the object type
    
    Parameters:
    
    original - the original supplier
    
    Returns:
    
    the supplier with memorization
  - partition
```
public static <T> List<List<T>> partition(List<T> list,
                                          int numLists)
```
    Partitions a list into numLists many lists each with around list.size() / numLists elements.
    
    Type Parameters:
    
    T - the object type
    
    Parameters:
    
    list - the list to partition
    
    numLists - number of lists to return
    
    Returns:
    
    partitioned list
  - isAddressReachable
```
public static boolean isAddressReachable(String hostname,
                                         int port,
                                         int timeoutMs)
```
    Validates whether a network address is reachable.
    
    Parameters:
    
    hostname - host name of the network address
    
    port - port of the network address
    
    timeoutMs - duration to attempt connection before returning false
    
    Returns:
    
    whether the network address is reachable
  - recursiveListLocalDir
```
public static List<File> recursiveListLocalDir(File dir)
```
    Recursively lists a local dir and all its subdirs and return all the files.
    
    Parameters:
    
    dir - the directory
    
    Returns:
    
    a list of all the files
  - parseMajorVersion
```
public static int parseMajorVersion(String version)
```
    Parameters:
    
    version - the version string of the JVMgi
    
    Returns:
    
    the major version of the current JVM, 8 for 1.8, 11 for java 11 see https://www.oracle.com/java/technologies/javase/versioning-naming.html for reference

Class CommonUtils

Nested Class Summary

Field Summary

Method Summary

Methods inherited from class java.lang.Object

Field Detail

PROCESS_TYPE

Method Detail

createProgressThread

createProgressThread

getCurrentMs

getJavaVersion

getTmpDir

getWorkerDataDirectory

listToString

argsToString

toStringArray

randomAlphaNumString

randomBytes

sleepMs

sleepMs

warmUpLoop

createNewClassInstance

getUnixGroups

waitFor

waitFor

waitForResult

waitForResult

getPrimaryGroupName

getGroups

stripSuffixIfPresent

stripPrefixIfPresent

stripLeadingAndTrailingQuotes

getValueFromStaticMapping

getRootCause

castToIOException

singleElementIterator

invokeAll

invokeAll

closeAndRethrow

closeAndRethrowRuntimeException

unwrapResponse

unwrapResponseFrom

isLocalHost

convertMsToDate

convertMsToClockTime

convertToByteString

convertFromByteString

memoize

partition

isAddressReachable

recursiveListLocalDir

parseMajorVersion