Configuration

Slack Docker Pulls

In this documentation we focus on the content, please see deployment instruction for where to put them.

Alluxio Site Properties

Alluxio Edge can be configured by setting the values of configuration properties within alluxio-site.properties.

License configurations

Alluxio Edge requires a license and an ETCD cluster to connect to. These are configured by the following properties.

alluxio.license=<YOUR_LICENSE_STRING>
# list the URLs for each etcd host in the etcd cluster, ex. http://trino-edge-etcd1:2379,http://trino-edge-etcd2:2379,http://trino-edge-etcd3:2379
alluxio.etcd.endpoints=<YOUR ETCD ENDPOINTS HERE>

As needed, set the username and password to authenticate to ETCD.

alluxio.etcd.username=username
alluxio.etcd.password=password

Specify the Root Under File System

This sample code specifies AWS S3 as the root under file system that Alluxio Edge will be accessing.

The minimal configuration needed is to set credentials for S3 so that Alluxio Edge can access S3. Please update the values for s3a.accessKeyId and s3a.secretKey corresponding to your own S3 credentials mentioned in prerequisites.

# ex. alluxio.dora.client.ufs.root=s3a://myBucket
alluxio.dora.client.ufs.root=<YOUR_S3_URL>
s3a.accessKeyId=<MY_KEY_ID>
s3a.secretKey=<MY_SECRET_KEY>

If the environment is configured with Instance Profiles the secret key and access key properties are not needed.

(Optional) To connect to S3 from a different region.

# For S3 endpoints
alluxio.underfs.s3.endpoint=s3.<PUT_YOUR_AWS_REGION_HERE>.amazonaws.com
alluxio.underfs.s3.endpoint.region=<PUT_YOUR_AWS_REGION_HERE> # Example: us-east-1
# For S3 buckets
alluxio.underfs.s3.region=<PUT_YOUR_AWS_REGION_HERE> # Example: us-east-1

(Optional) To enable TLS in Alluxio Edge if server side encryption is on.

alluxio.underfs.s3.secure.http.enabled=true
alluxio.underfs.s3.server.side.encryption.enabled=true

(Optional) If assume role is enabled in S3.

alluxio.underfs.s3.assumerole.enabled=true
alluxio.underfs.s3.assumerole.rolearn=<PUT_YOUR_ASSUME_ROLE_ARN> # Example: arn:aws:iam::...

Specify the Cache Storage Directories

Trino worker and coordinator needs to know the Alluxio Edge cache size alluxio.user.client.cache.size and path alluxio.user.client.cache.dirs. This sample code specifies the cache storage directories Alluxio Edge will use.

# Enable edge cache on client (RAM disk only)
#
#alluxio.user.client.cache.enabled=true
#alluxio.user.client.cache.size=1GB
#alluxio.user.client.cache.dirs=/dev/shm/alluxio_cache

# Enable edge cache on client (example with 2 NVMe volumes)
#
alluxio.user.client.cache.enabled=true
alluxio.user.client.cache.size=600GB,600GB
alluxio.user.client.cache.dirs=/mnt/nvme0/alluxio_cache,/mnt/nvme1/alluxio_cache

(Optional) Enable Alluxio Edge metrics collection

#
alluxio.user.metrics.collection.enabled=true