alluxio.util

Class ConfigurationUtils

java.lang.Object

alluxio.util.ConfigurationUtils

public final class ConfigurationUtils
extends Object

Utilities for working with Alluxio configurations.

Method Summary

Modifier and Type	Method and Description
static float	checkRatio(AlluxioConfiguration conf, PropertyKey key) Checks that the given property key is a ratio from 0.0 and 1.0, throwing an exception if it is not.
static AlluxioProperties	defaults() Returns an instance of AlluxioConfiguration with the defaults and values from alluxio-site properties.
static AlluxioConfiguration	getClusterConf(GetConfigurationPResponse response, AlluxioConfiguration conf, Scope scope) Loads the cluster level configuration from the get configuration response, filters out the configuration for certain scope, and merges it with the existing configuration.
static List<ConfigProperty>	getConfiguration(AlluxioConfiguration conf, Scope scope) Gets all configuration properties filtered by the specified scope.
static List<InetSocketAddress>	getEmbeddedJournalAddresses(AlluxioConfiguration conf, NetworkAddressUtils.ServiceType serviceType) Gets the embedded journal addresses to use for the given service type (either master-raft or job-master-raft).
static List<InetSocketAddress>	getJobMasterEmbeddedJournalAddresses(AlluxioConfiguration conf)
static String	getJobMasterHostNotConfiguredMessage(String serviceName) Returns a unified message for cases when the job master hostname cannot be determined.
static List<InetSocketAddress>	getJobMasterRpcAddresses(AlluxioConfiguration conf) Gets the RPC addresses of all job masters based on the configuration.
static List<InetSocketAddress>	getMasterEmbeddedJournalAddresses(AlluxioConfiguration conf)
static Set<String>	getMasterHostnames(AlluxioConfiguration conf) Gets list of masters in conf directory.
static String	getMasterHostNotConfiguredMessage(String serviceName) Returns a unified message for cases when the master hostname cannot be determined.
static List<InetSocketAddress>	getMasterRpcAddresses(AlluxioConfiguration conf) Gets the RPC addresses of all masters based on the configuration.
static PathConfiguration	getPathConf(GetConfigurationPResponse response, AlluxioConfiguration clusterConf) Loads the path level configuration from the get configuration response.
static String	getSchemeAuthority(AlluxioConfiguration conf)
static Set<String>	getServerHostnames(AlluxioConfiguration conf) Gets list of masters/workers in conf directory.
static Set<String>	getWorkerHostnames(AlluxioConfiguration conf) Gets list of workers in conf directory.
static boolean	isHaMode(AlluxioConfiguration conf)
static boolean	jobMasterHostConfigured(AlluxioConfiguration conf)
static GetConfigurationPResponse	loadConfiguration(InetSocketAddress address, AlluxioConfiguration conf, boolean ignoreClusterConf, boolean ignorePathConf) Loads configuration from meta master in one RPC.
static Properties	loadProperties(InputStream stream)
static Properties	loadPropertiesFromFile(String filePath) Loads properties from the given file.
static Properties	loadPropertiesFromResource(URL resource) Loads properties from a resource.
static boolean	masterHostConfigured(AlluxioConfiguration conf)
static AlluxioConfiguration	merge(AlluxioConfiguration conf, Map<?,?> properties, Source source) Merges the current configuration properties with new properties.
static List<String>	parseAsList(String value, String delimiter) Returns the input string as a list, splitting on a specified delimiter.
static void	reloadProperties() Reloads site properties from disk.
static String	searchPropertiesFile(String propertiesFile, String[] confPathList) Searches the given properties file from a list of paths.
static String	valueAsString(String value)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Method Detail

getEmbeddedJournalAddresses

public static List<InetSocketAddress> getEmbeddedJournalAddresses(AlluxioConfiguration conf,
                                                                  NetworkAddressUtils.ServiceType serviceType)

Gets the embedded journal addresses to use for the given service type (either master-raft or job-master-raft).

Parameters:

conf - configuration

serviceType - the service to get addresses for

Returns:

the addresses

getMasterEmbeddedJournalAddresses

public static List<InetSocketAddress> getMasterEmbeddedJournalAddresses(AlluxioConfiguration conf)

Parameters:

conf - configuration

Returns:

the embedded journal addresses to use for the master

getJobMasterEmbeddedJournalAddresses

public static List<InetSocketAddress> getJobMasterEmbeddedJournalAddresses(AlluxioConfiguration conf)

Parameters:

conf - configuration

Returns:

the embedded journal addresses to use for the job master

getMasterRpcAddresses

public static List<InetSocketAddress> getMasterRpcAddresses(AlluxioConfiguration conf)

Gets the RPC addresses of all masters based on the configuration.

Parameters:

conf - the configuration to use

Returns:

the master rpc addresses

getJobMasterRpcAddresses

public static List<InetSocketAddress> getJobMasterRpcAddresses(AlluxioConfiguration conf)

Gets the RPC addresses of all job masters based on the configuration.

Parameters:

conf - the configuration to use

Returns:

the job master rpc addresses

loadPropertiesFromResource

@Nullable
public static Properties loadPropertiesFromResource(URL resource)

Loads properties from a resource.

Parameters:

resource - url of the properties file

Returns:

a set of properties on success, or null if failed

loadPropertiesFromFile

@Nullable
public static Properties loadPropertiesFromFile(String filePath)

Loads properties from the given file.

Parameters:

filePath - the absolute path of the file to load properties

Returns:

a set of properties on success, or null if failed

loadProperties

@Nullable
public static Properties loadProperties(InputStream stream)

Parameters:

stream - the stream to read properties from

Returns:

a properties object populated from the stream

searchPropertiesFile

@Nullable
public static String searchPropertiesFile(String propertiesFile,
                                                    String[] confPathList)

Searches the given properties file from a list of paths.

Parameters:

propertiesFile - the file to load properties

confPathList - a list of paths to search the propertiesFile

Returns:

the site properties file on success search, or null if failed

jobMasterHostConfigured

public static boolean jobMasterHostConfigured(AlluxioConfiguration conf)

Parameters:

conf - the configuration to use

Returns:

whether the configuration describes how to find the job master host, either through explicit configuration or through zookeeper

getMasterHostNotConfiguredMessage

public static String getMasterHostNotConfiguredMessage(String serviceName)

Returns a unified message for cases when the master hostname cannot be determined.

Parameters:

serviceName - the name of the service that couldn't run. i.e. Alluxio worker, fsadmin shell, etc.

Returns:

a string with the message

getJobMasterHostNotConfiguredMessage

public static String getJobMasterHostNotConfiguredMessage(String serviceName)

Returns a unified message for cases when the job master hostname cannot be determined.

Parameters:

serviceName - the name of the service that couldn't run. i.e. Alluxio worker, fsadmin shell, etc.

Returns:

a string with the message

checkRatio

public static float checkRatio(AlluxioConfiguration conf,
                               PropertyKey key)

Checks that the given property key is a ratio from 0.0 and 1.0, throwing an exception if it is not.

Parameters:

conf - the configuration for looking up the property key

key - the property key

Returns:

the property value

masterHostConfigured

public static boolean masterHostConfigured(AlluxioConfiguration conf)

Parameters:

conf - the configuration to use

Returns:

whether the configuration describes how to find the master host, either through explicit configuration or through zookeeper

isHaMode

public static boolean isHaMode(AlluxioConfiguration conf)

Parameters:

conf - the configuration use

Returns:

whether the configuration specifies to run in ha mode

getConfiguration

public static List<ConfigProperty> getConfiguration(AlluxioConfiguration conf,
                                                    Scope scope)

Gets all configuration properties filtered by the specified scope.

Parameters:

conf - the configuration to use

scope - the scope to filter by

Returns:

the properties

valueAsString

public static String valueAsString(String value)

Parameters:

value - the value or null (value is not set)

Returns:

the value or "(no value set)" when the value is not set

defaults

public static AlluxioProperties defaults()

Returns an instance of AlluxioConfiguration with the defaults and values from alluxio-site properties.

Returns:

the set of Alluxio properties loaded from the site-properties file

reloadProperties

public static void reloadProperties()

Reloads site properties from disk.

merge

public static AlluxioConfiguration merge(AlluxioConfiguration conf,
                                         Map<?,?> properties,
                                         Source source)

Merges the current configuration properties with new properties. If a property exists both in the new and current configuration, the one from the new configuration wins if its priority is higher or equal than the existing one.

Parameters:

conf - the base configuration

properties - the source Properties to be merged

source - the source of the the properties (e.g., system property, default and etc)

Returns:

a new configuration representing the merged properties

loadConfiguration

public static GetConfigurationPResponse loadConfiguration(InetSocketAddress address,
                                                          AlluxioConfiguration conf,
                                                          boolean ignoreClusterConf,
                                                          boolean ignorePathConf)
                                                   throws AlluxioStatusException

Loads configuration from meta master in one RPC.

Parameters:

address - the meta master address

conf - the existing configuration

ignoreClusterConf - do not load cluster configuration related information

ignorePathConf - do not load path configuration related information

Returns:

the RPC response

Throws:

AlluxioStatusException

getClusterConf

public static AlluxioConfiguration getClusterConf(GetConfigurationPResponse response,
                                                  AlluxioConfiguration conf,
                                                  Scope scope)

Loads the cluster level configuration from the get configuration response, filters out the configuration for certain scope, and merges it with the existing configuration.

Parameters:

response - the get configuration RPC response

conf - the existing configuration

scope - the target scope

Returns:

the merged configuration

getPathConf

public static PathConfiguration getPathConf(GetConfigurationPResponse response,
                                            AlluxioConfiguration clusterConf)

Loads the path level configuration from the get configuration response. Only client scope properties will be loaded.

Parameters:

response - the get configuration RPC response

clusterConf - cluster level configuration

Returns:

the loaded path level configuration

getSchemeAuthority

public static String getSchemeAuthority(AlluxioConfiguration conf)

Parameters:

conf - the configuration

Returns:

the alluxio scheme and authority determined by the configuration

parseAsList

public static List<String> parseAsList(String value,
                                       String delimiter)

Returns the input string as a list, splitting on a specified delimiter.

Parameters:

value - the value to split

delimiter - the delimiter to split the values

Returns:

the list of values for input string

getMasterHostnames

public static Set<String> getMasterHostnames(AlluxioConfiguration conf)

Gets list of masters in conf directory.

Parameters:

conf - configuration

Returns:

master hostnames

getWorkerHostnames

public static Set<String> getWorkerHostnames(AlluxioConfiguration conf)

Gets list of workers in conf directory.

Parameters:

conf - configuration

Returns:

workers hostnames

getServerHostnames

public static Set<String> getServerHostnames(AlluxioConfiguration conf)

Gets list of masters/workers in conf directory.

Parameters:

conf - configuration

Returns:

server hostnames