Configuring Alluxio with MapR-FS

Slack Docker Pulls

This guide describes how to configure Alluxio with MapR-FS as the under storage system.

Configuring Alluxio for MapR-FS

By default, Alluxio is already configured to access MapR-FS URIs. The configuration parameter that enables access to MapR-FS URIs is alluxio.underfs.hdfs.prefixes. Since maprfs:/// is already included in that parameter by default, no futher configuration is necessary.

Mounting MapR-FS

Alluxio unifies access to different storage systems through the unified namespace feature. A MapR-FS location can be either mounted at the root of the Alluxio namespace or at a nested directory.

Root Mount

When installing Alluxio, the under storage address should be specified in conf/alluxio-site.properties.

alluxio.underfs.address=maprfs:///<folder in MapR-FS>/
alluxio.master.mount.table.root.option.alluxio.underfs.hdfs.version=maprfs-5.2

It is recommended to use a folder like maprfs:///user/alluxio for a fresh installation. Ensure that the user launching Alluxio has the appropriate permissions in MapR-FS to the folder specified.

Nested Mount

A MapR-FS location can be mounted at a nested directory in the Alluxio namespace to have unified access to multiple under storage systems. Alluxio’s Command Line Interface can be used for this purpose.

$ ./bin/alluxio fs mount --option alluxio.underfs.hdfs.version=maprfs-5.2 /mnt/ maprfs:///<folder in MapR-FS>/

Running Alluxio with MapR-FS

After configuring Alluxio with a MapR-FS Under File System, follow the rest of the Alluxio installation steps to complete the cluster deployment. To run basic I/O tests, using the Alluxio Command Line Interface.

$ ./bin/alluxio runTests

If testing a nested mount point, run:

$ ./bin/alluxio runTests --directory /mnt/

After this succeeds, view MapR-FS location to verify the files and directories created by Alluxio. For this test, you should see files named like:

$ /user/alluxio/default_tests_files/BASIC_CACHE_THROUGH