Alluxio Community Day VIII

Join us at our next virtual community event on December 14th featuring fellow Alluxio community users from Apache Iceberg and WeRide.


Slack Docker Pulls GitHub edit source

This guide describes how to configure Alluxio with CephFS as the under storage system. Alluxio supports two different implementations of under storage system for CephFS:


Deploy Alluxio binary package

The Alluxio binaries must be on your machine. You can either compile Alluxio, or download the binaries locally.

Install Dependences

According to ceph packages install to install below packages:

$ ln -s /usr/lib64/ /usr/lib64/
$ ln -s /usr/lib64/ /usr/lib64/
$ java_path=`which java | xargs readlink | sed 's#/bin/java##g'`
$ ln -s /usr/share/java/libcephfs.jar $java_path/jre/lib/ext/libcephfs.jar

Download CephFS Hadoop jar

$ curl -o $java_path/jre/lib/ext/hadoop-cephfs.jar -s

Basic Setup

Configure Alluxio to use under storage systems by modifying conf/ and conf/core-site.xml. If them do not exist, create the configuration files from the templates

$ cp conf/ conf/
$ cp conf/core-site.xml.template conf/core-site.xml

Running Alluxio Locally with CephFS

Start up Alluxio locally to see that everything works.

$ ./bin/alluxio format
$ ./bin/ local

This should start an Alluxio master and Alluxio worker. You can see the master UI at http://localhost:19999.