Building Alluxio Master Branch

Slack Docker Pulls GitHub edit source

This guide describes how to compile Alluxio from the beginning.

The prerequisite for this guide is that you have Java 7 (or above), Maven, and Thrift 0.9.3 (Optional) installed.

Checkout the Alluxio master branch from Github and package:

git clone git://
cd alluxio
mvn install -DskipTests

If you are seeing java.lang.OutOfMemoryError: Java heap space, please execute:

export MAVEN_OPTS="-Xmx2g -XX:MaxPermSize=512M -XX:ReservedCodeCacheSize=512m"

If you want to build a particular version of Alluxio, for example 1.4.0, please do git checkout v1.4.0 after cd alluxio.

The Maven build system fetches its dependencies, compiles source code, runs unit tests, and packages the system. If this is the first time you are building the project, it can take a while to download all the dependencies. Subsequent builds, however, will be much faster.

Once Alluxio is built, you can start it with:

./bin/alluxio format
./bin/ local

To verify that Alluxio is running, you can visit http://localhost:19999 or check the log in the alluxio/logs directory. You can also run a simple program:

./bin/alluxio runTests

You should be able to see results similar to the following:

/default_tests_files/BasicFile_STORE_SYNC_PERSIST has been removed
2015-10-20 23:02:54,403 INFO   ( - Alluxio client (version 1.0.0) is trying to connect with FileSystemMaster master @ localhost/
2015-10-20 23:02:54,422 INFO   ( - Client registered with FileSystemMaster master @ localhost/
2015-10-20 23:02:54,460 INFO   ( - createFile with fileId 1476395007 took 65 ms.
2015-10-20 23:02:54,557 INFO   ( - Alluxio client (version 1.0.0) is trying to connect with BlockMaster master @ localhost/
2015-10-20 23:02:54,558 INFO   ( - Client registered with BlockMaster master @ localhost/
2015-10-20 23:02:54,590 INFO   ( - Connecting local worker @ /
2015-10-20 23:02:54,654 INFO   ( - Folder /Volumes/ramdisk/alluxioworker/6601007274872912185 was created!
2015-10-20 23:02:54,657 INFO   (<init>) - LocalBlockOutStream created new file block, block path: /Volumes/ramdisk/alluxioworker/6601007274872912185/1459617792
2015-10-20 23:02:54,658 INFO   ( - Connecting local worker @ /
2015-10-20 23:02:54,754 INFO   ( - writeFile to file /default_tests_files/BasicFile_STORE_SYNC_PERSIST took 294 ms.
2015-10-20 23:02:54,803 INFO   ( - readFile file /default_tests_files/BasicFile_STORE_SYNC_PERSIST took 47 ms.
Passed the test!

You can also stop the system by using:

./bin/ all

Unit Tests

To run all unit tests:

mvn test

To run all the unit tests with under storage other than local filesystem:

mvn test [ -Dhadoop.version=x.x.x ] [ -P<under-storage-profile> ]

Currently supported values for <under-storage-profile> are:

Not Specified # [Default] Tests against local file system
swiftTest     # Tests against a simulated Swift cluster
hdfsTest      # Tests against HDFS minicluster
glusterfsTest # Tests against GlusterFS
s3Test        # Tests against Amazon S3 (requires a real S3 bucket)
ossTest       # Tests against Aliyun OSS (requires a real OSS bucket)
gcsTest       # Tests against Google Cloud Storage (requires a real GCS bucket)

To have the logs output to STDOUT, append the following to the mvn command

-Dtest.output.redirect=false -Dalluxio.root.logger=DEBUG,CONSOLE

Distro Support

To build Alluxio against one of the different distros of hadoop, you only need to change the hadoop.version.


All main builds are from Apache so all Apache releases can be used directly



To build against Cloudera’s releases, just use a version like $apacheRelease-cdh$cdhRelease



To build against a MapR release



To build against a Pivotal release, just use a version like $apacheRelease-gphd-$pivotalRelease



To build against a Hortonworks release, just use a version like $apacheRelease.$hortonRelease


System Settings

Sometimes you will need to play with a few system settings in order to have the unit tests pass locally. A common setting that may need to be set is ulimit.


In order to increase the number of files and processes allowed, run the following

sudo launchctl limit maxfiles 16384 16384
sudo launchctl limit maxproc 2048 2048

It is also recommended to exclude your local clone of Alluxio from Spotlight indexing. Otherwise, your Mac may hang constantly trying to re-index the file system during the unit tests. To do this, go to System Preferences > Spotlight > Privacy, click the + button, browse to the directory containing your local clone of Alluxio, and click Choose to add it to the exclusions list.