TLS Encryption
TLS (Transport Layer Security) is a cryptographic protocol that ensures secure communication over the internet. This guide provides detailed instructions on configuring TLS support in Alluxio to secure RPCs and data transfers. Note that, while ensuring data integrity and confidentiality during transmission, enabling TLS may also introduce performance overhead in data transmission.
Enable TLS Encryption
To configure Alluxio for TLS encryption, you need to set up both keystores and truststores.
Keystore and Truststore Overview
- Keystore: Manages private keys and certificates to identify the entity during TLS handshaking.
- Truststore: Manages certificates to verify certificates received during TLS handshaking.
Setup Keystore
Alluxio servers (workers and coordinators) require a keystore to enable TLS. The keystore holds the private key and certificate for the server. Ensure that the keystore file is accessible to the OS user running the Alluxio server processes.
Create a self-signed keystore using the following command:
keytool -genkeypair -alias key -keyalg RSA -keysize 2048 \
-dname "CN=localhost, OU=Department, O=Company, L=City, ST=State, C=US" \
-keystore /alluxio/keystore.jks -keypass keypass -storepass keypass
This command generates a keystore at /alluxio/keystore.jks
with both the key password and keystore password set to keypass
.
Setup Truststore
All clients involved in a TLS connection require a truststore to trust the certificates provided by the servers. Clients in Alluxio include Alluxio clients, workers (which communicate with the coordinator), and the coordinator itself (which communicates with workers). The truststore must be readable by the processes initiating the connections.
Create a truststore using the keystore from the previous step:
keytool -export -alias key -keystore /alluxio/keystore.jks \
-storepass keypass -rfc -file selfsigned.cer
keytool -import -alias key -noprompt -file selfsigned.cer \
-keystore /alluxio/truststore.jks -storepass trustpass
- The first command extracts the certificate from the keystore using the password
keypass
. - The second command imports this certificate into a truststore at
/alluxio/truststore.jks
with the passwordtrustpass
.
Configure Alluxio Servers
After setting up the keystore and truststore, add the following properties to alluxio-site.properties
on Alluxio servers:
# Enable TLS
alluxio.network.tls.enabled=true
# Keystore configuration
alluxio.network.tls.keystore.path=/alluxio/keystore.jks
alluxio.network.tls.keystore.password=keypass
alluxio.network.tls.keystore.key.password=keypass
# Truststore configuration
alluxio.network.tls.truststore.path=/alluxio/truststore.jks
alluxio.network.tls.truststore.password=trustpass
# Worker transfer type
alluxio.worker.network.netty.file.transfer=MAPPED
Note: Enabling TLS on workers requires setting alluxio.worker.network.netty.file.transfer
to MAPPED
, which may affect performance due to the disabling of Netty zero-copy.
Advanced Settings
- Specify TLS Protocols: To restrict the server to certain TLS protocols, set:
alluxio.network.tls.server.protocols=TLSv1.2,TLSv1.3
. - Multiple Keys in Keystore: If multiple keys exist, specify the key alias:
alluxio.network.tls.keystore.alias=serverkey
. - Disable Client Hostname Verification: For users who need flexible access from clients to the service, one can disable hostname verification:
alluxio.network.tls.client.no.endpoint.identification=true
.
Configure Alluxio Clients
Add the following properties to alluxio-site.properties
on Alluxio clients:
# Enable TLS for clients
alluxio.network.tls.enabled=true
# Truststore configuration for clients
alluxio.network.tls.truststore.path=/alluxio/truststore.jks
alluxio.network.tls.truststore.password=trustpass
alluxio.network.tls.client.no.endpoint.identification=true
After these configurations, all network communication with Alluxio will be encrypted using TLS.
Configure Connection to ETCD with TLS
If your Alluxio cluster uses ETCD and requires TLS encryption for the connection, check out the section in use external etcd.
Enable TLS Encryption for Alluxio in Kubernetes
Enabling TLS in Kubernetes differs due to the ephemeral nature of pods. Follow these steps to enable TLS encryption for Alluxio in a Kubernetes environment.
Generate Keypair
Use keytool
to generate a keystore and truststore pair as described in the Setup Keystore and Setup Truststore sections. In Kubernetes, it’s common to disable client-side hostname verification because pod hostnames can change.
Create Kubernetes Secrets
Create Kubernetes secrets using the generated keystore and truststore:
$ kubectl create secret generic alluxio-tls-keystore --from-file=/alluxio/keystore.jks
$ kubectl create secret generic alluxio-tls-truststore --from-file=/alluxio/truststore.jks
Mount Secrets to Pods
When installing Alluxio using the operator, configure the secrets in the alluxio-cluster.yaml
file under the spec.secrets
section:
spec:
secrets:
coordinator:
alluxio-tls-keystore: /secrets/alluxio-tls-keystore
alluxio-tls-truststore: /secrets/alluxio-tls-truststore
worker:
alluxio-tls-keystore: /secrets/alluxio-tls-keystore
alluxio-tls-truststore: /secrets/alluxio-tls-truststore
This mounts the secrets into the pods at the specified paths.
Configure Alluxio Properties in Kubernetes
Set the Alluxio properties in your alluxio-cluster.yaml
file under the spec.properties
section:
spec:
properties:
alluxio.network.tls.enabled: "true"
alluxio.network.tls.keystore.path: /secrets/alluxio-tls-keystore/keystore.jks
alluxio.network.tls.keystore.password: keypass
alluxio.network.tls.keystore.alias: key
alluxio.network.tls.keystore.key.password: keypass
alluxio.network.tls.truststore.path: /secrets/alluxio-tls-truststore/truststore.jks
alluxio.network.tls.truststore.password: trustpass
alluxio.network.tls.truststore.alias: key
alluxio.network.tls.client.no.endpoint.identification: "true"
alluxio.worker.network.netty.file.transfer: "MAPPED"
Examples
Configure Spark
To configure Spark with an Alluxio TLS-enabled client, set the following properties in spark-defaults.conf
:
spark.driver.extraJavaOptions=-Dalluxio.network.tls.enabled=true -Dalluxio.network.tls.truststore.path=<TRUSTSTORE_PATH> -Dalluxio.network.tls.truststore.password=<TRUSTSTORE_PASSWORD> -Dalluxio.network.tls.client.no.endpoint.identification=true
spark.executor.extraJavaOptions=-Dalluxio.network.tls.enabled=true -Dalluxio.network.tls.truststore.path=<TRUSTSTORE_PATH> -Dalluxio.network.tls.truststore.password=<TRUSTSTORE_PASSWORD> -Dalluxio.network.tls.client.no.endpoint.identification=true
Replace <TRUSTSTORE_PATH>
and <TRUSTSTORE_PASSWORD>
with your truststore’s path and password.
Note: If you encounter errors like OPENSSL_internal:SSLV3_ALERT_HANDSHAKE_FAILURE
, specify the TLS protocol version:
alluxio.network.tls.server.protocols=TLSv1.3
Configure Trino in Kubernetes
To enable TLS when Trino accesses an Alluxio cluster in Kubernetes, follow these steps:
Mount Kubernetes Secrets to Trino
After creating the secrets for the keystore and truststore, mount them to the Trino pods. If you’re using Trino’s official Helm chart, add the following to your values.yaml
file:
secretMounts:
- name: alluxio-tls-truststore
secretName: alluxio-tls-truststore
path: /secrets/alluxio-tls-truststore
Configure TLS Properties in Trino
Add the TLS-related Alluxio properties to Trino’s configuration:
alluxio.network.tls.enabled=true
alluxio.network.tls.truststore.path=/secrets/alluxio-tls-truststore/truststore.jks
alluxio.network.tls.truststore.password=trustpass
alluxio.network.tls.truststore.alias=key
alluxio.network.tls.client.no.endpoint.identification=true
These properties should be added to both the Trino coordinator and worker configurations. communication between clients and servers in both traditional and Kubernetes environments.