Configuring Alluxio with Azure Blob Store
This guide describes how to configure Alluxio with Azure Blob Store as the under storage system.
In preparation for using Azure Blob Store with Alluxio, create a new container in your Azure
storage account or use an existing container. You should also note that the directory you want to
use in that container, either by creating a new directory in the container, or using an existing
one. For the purposes of this guide, the Azure storage account name is called
container in that storage account is called
<AZURE_CONTAINER> and the directory in that container is
<AZURE_DIRECTORY>. For more information about Azure storage account, Please see
Mounting Azure Blob Store
An Azure Blob Store location can be mounted either at the root of the Alluxio namespace or at a nested directory. Alluxio uses the Hadoop support for Azure blob storage (wasb).
When installing Alluxio, the following properties should be specified in
alluxio.underfs.address=wasb://<AZURE_CONTAINER>@<AZURE_ACCOUNT>.blob.core.windows.net/<AZURE_DIRECTORY>/ alluxio.master.mount.table.root.option.fs.azure.account.key.<AZURE_ACCOUNT>.blob.core.windows.net=<YOUR ACCESS KEY>
An Azure blob store location can be mounted at a nested directory in the Alluxio namespace to have unified access to multiple under storage systems. Alluxio’s Command Line Interface can be used for this purpose.
$ ./bin/alluxio fs mount --option fs.azure.account.key.<AZURE_ACCOUNT>.blob.core.windows.net=<AZURE_ACCESS_KEY> \ /mnt/azure wasb://<AZURE_CONTAINER>@<AZURE_ACCOUNT>.blob.core.windows.net/<AZURE_DIRECTORY>/
Running Alluxio Locally with Azure Blob Store
After configuring Alluxio with Azure Blob Store, follow the rest of the Alluxio installation steps to complete the cluster deployment. To run basic I/O tests, click into the deployed cluster and use the drop down menu in the top right. All the tests should pass if the configurations and setup are correct.
Otherwise, tests can be run using the Alluxio Command Line Interface.
$ ./bin/alluxio runTests
If testing a nested mount point, run:
$ ./bin/alluxio runTests --directory /mnt/
After this succeeds, you can visit your container
<AZURE_CONTAINER> to verify the files and directories created by Alluxio exist. For this test, you should see files named like: