Azure Blob Store

Slack Docker Pulls

This guide describes how to configure Alluxio with Azure Blob Store as the under storage system.

Azure Blob Storage is Microsoft’s object storage solution for the cloud. Blob Storage is optimized for storing massive amounts of unstructured data.

For more information about Azure Blob Store, please read its documentation.

Prerequisites

If you haven’t already, please see Prerequisites before you get started.

In preparation for using Azure Blob Store with Alluxio:

<AZURE_CONTAINER> Create a new container in your Azure storage account or use an existing container.
<AZURE_DIRECTORY> The directory you want to use in that container, either by creating a new directory or using an existing one
<AZURE_ACCOUNT> Your Azure storage account
<AZURE_ACCOUNT_KEY> Your Azure account credientials

Basic Setup

To use Azure Blob Store as the UFS of Alluxio root mount point, you need to configure Alluxio to use under storage systems by modifying conf/alluxio-site.properties. If it does not exist, create the configuration file from the template.

$ cp conf/alluxio-site.properties.template conf/alluxio-site.properties

Specify the underfs address by modifying conf/alluxio-site.properties to include:

alluxio.dora.client.ufs.root=wasbs://<AZURE_CONTAINER>@<AZURE_ACCOUNT>.blob.core.windows.net/<AZURE_DIRECTORY>/

Specify credentials for the Azure account of the root mount point by adding the following properties in conf/alluxio-site.properties:

fs.azure.account.key.<AZURE_ACCOUNT>.blob.core.windows.net=<AZURE_ACCOUNT_KEY>

Running Alluxio Locally with Azure Blob Store

Once you have configured Alluxio to Azure Blob Store, try running Alluxio locally to see that everything works.

Troubleshooting

What should I do if I get http not support error?

If you mount the Blob and configure the Blob path start with wasb://, you may see the error as follows:

alluxio.exception.AlluxioException: com.microsoft.azure.storage.StorageException: The account being accessed does not support http.

You can change the Blob path start with wasbs://.