MapR-FS
- Compiling Alluxio with MapR Version
- Configuring Alluxio for MapR-FS
- Configuring Alluxio to use MapR-FS as Under Storage
- Running Alluxio Locally with MapR-FS
This guide describes how to configure Alluxio with MapR-FS as the under storage system.
Compiling Alluxio with MapR Version
Alluxio must be compiled
with the correct MapR distribution to integrate with MapR-FS. Here are some values of
hadoop.version for different MapR distributions:
| MapR Version | hadoop.version |
|---|---|
| 5.2 | 2.7.0-mapr-1607 |
| 5.1 | 2.7.0-mapr-1602 |
| 5.0 | 2.7.0-mapr-1506 |
| 4.1 | 2.5.1-mapr-1503 |
| 4.0.2 | 2.5.1-mapr-1501 |
| 4.0.1 | 2.4.1-mapr-1408 |
Configuring Alluxio for MapR-FS
Once you have compiled Alluxio with the appropriate hadoop.version for your MapR distribution, you
may have to configure Alluxio to recognize the MapR-FS scheme and URIs. Alluxio uses the HDFS client
to access MapR-FS, and by default is already configured to do so. However, if the configuration has
been changed, you can enable the HDFS client to access MapR-FS URIs by adding the URI prefix
maprfs:/// to the configuration variable alluxio.underfs.hdfs.prefixes like below:
alluxio.underfs.hdfs.prefixes=hdfs://,maprfs:///
This configuration parameter should be set for all the Alluxio servers (masters, workers). Please
read how to configure Alluxio. For
Alluxio processes, this parameter can be set in the property file alluxio-site.properties. For
more information, please read about
configuration of Alluxio with property files.
Configuring Alluxio to use MapR-FS as Under Storage
There are various ways to configure Alluxio to use MapR-FS as under storage. If you want to
mount MapR-FS to the root of Alluxio, add the following to conf/alluxio-site.properties:
alluxio.underfs.address=maprfs:///<path in MapR-FS>/
You can also mount a directory in MapR-FS to a directory in the Alluxio namespace.
${ALLUXIO_HOME}/bin/alluxio fs mount /<path in Alluxio>/ maprfs:///<path in MapR-FS>/
Running Alluxio Locally with MapR-FS
Start up Alluxio locally to see that everything works.
./bin/alluxio format
./bin/alluxio-start.sh local
This should start one Alluxio master and one Alluxio worker locally. You can see the master UI at http://localhost:19999.
Run a simple example program:
./bin/alluxio runTests
Visit MapR-FS web UI to verify the files and directories created by
Alluxio exist. For this test, you should see files named like:
/default_tests_files/BASIC_CACHE_CACHE_THROUGH
Stop Alluxio by running:
./bin/alluxio-stop.sh local