Running Alluxio on a Cluster

Slack Docker Pulls GitHub edit source

Download Alluxio

First download the Alluxio tar file, and extract it.

wget http://alluxio.org/downloads/files/1.5.0/alluxio-1.5.0-bin.tar.gz
tar xvfz alluxio-1.5.0-bin.tar.gz

Configure Alluxio

In the ${ALLUXIO_HOME}/conf directory, create the conf/alluxio-site.properties configuration file from the template.

cp conf/alluxio-site.properties.template conf/alluxio-site.properties

Update alluxio.master.hostname in conf/alluxio-site.properties to the hostname of the machine you plan to run Alluxio Master on. Add the IP addresses of all the worker nodes to the conf/workers file. Finally, sync all the information to worker nodes. You can use

./bin/alluxio copyDir <dirname>

to sync files and folders to all hosts specified in the alluxio/conf/workers file.

Start Alluxio

Now, you can start Alluxio:

cd alluxio
./bin/alluxio format
./bin/alluxio-start.sh # use the right parameters here. e.g. all Mount
# Notice: the Mount and SudoMount parameters will format the existing RamFS.

To verify that Alluxio is running, you can visit http://<alluxio_master_hostname>:19999, check the log in the directory alluxio/logs, or run a sample program:

./bin/alluxio runTests

Note: If you are using EC2, make sure the security group settings on the master node allows incoming connections on the alluxio web UI port.