TLS Encryption on Alluxio in Kubernetes
- Setup TLS for Alluxio Cluster
- TLS termination at Proxy
Alluxio supports encryption of the network communication between services with TLS. Refer here for more information about the general usage of TLS.
Enabling TLS on Alluxio in Kubernetes is to some extent different from bare metal machines because of the nature of kubernetes that pods, where the services run, are ephemeral. Since the keystore needs to include the IP address or DNS of the server but a restarted pod doesn’t keep the old IP/DNS, we have to leverage the Kubernetes Service to ensure the restarted pod is reachable/ can reach other components of the system.
Setup TLS for Alluxio Cluster
To enable TLS encryption on Alluxio network traffic on Kubernetes, follow the steps below:
1. Generate keypair
keytool to generate a keystore/truststore pair, as described in here
Note: the generated keystore needs to contain the ip/hostname of all Alluxio masters/workers, otherwise the verification would fail.
The hostnames of Alluxio master pods follow the pattern
alluxio-master-x, where x ranges from 0 to # of master pods-1.
The hostnames of Alluxio worker pods follow the pattern
alluxio-worker-x, where x ranges from 0 to # of worker pods-1.
Therefore, you may need to append the SAN (Subject Alternative Name) extension to the keystore generation command. For example,
$ keytool -genkeypair ... -ext "SAN=DNS:alluxio-master-0,DNS:alluxio-worker-0,DNS:alluxio-worker-1,DNS:alluxio-worker-2"
2. Create Kubernetes secrets of the keypair
Create secrets in Kubernetes with the created keystore and truststore. For example,
$ kubectl create secret generic alluxio-tls-keystore --from-file=/alluxio/keystore.jks $ kubectl create secret generic alluxio-tls-truststore --from-file=/alluxio/truststore.jks
3. Configure the secrets for the pods
Configure the secrets in helm chart
values.yaml file under
.secrets section. For example,
secrets: master: alluxio-tls-keystore: alluxio-tls-keystore alluxio-tls-truststore: alluxio-tls-truststore worker: alluxio-tls-keystore: alluxio-tls-keystore alluxio-tls-truststore: alluxio-tls-truststore
4. Configure the keypair for Alluxio
Configure the properties for Alluxio in your
similar to here.
By default, the secrets are mounted under
/secrets/<secret-name>/<file-name>. Configure the keystore path and
the truststore path accordingly. For example,
properties: ... alluxio.network.tls.keystore.path: "/secrets/alluxio-tls-keystore/keystore.jks" alluxio.network.tls.truststore.path: "/secrets/alluxio-tls-truststore/truststore.jks" ...
5. Configure Alluxio worker deployment
To make sure the worker pod still works with the existing keystore/truststore after restart, Statefulset with service
is required for worker deployment, instead of the default Daemonset deployment type. To use Statefulset, set
true in your
values.yaml file, and set the number of worker pods to spawn. For example,
worker: useStatefulset: true count: 3 # <---- number of worker pods you need
6. Deploy Alluxio and check TLS is enabled
Deploy Alluxio Cluster via
helm install. To make sure TLS is enabled, check the logs of the respective services.
Similar lines should be logged which indicate the communication iss encrypted by TLS:
2022-10-31 00:53:26,306 INFO EnterpriseSslContextProvider - TLS server using keystore: /secrets/alluxio-tls-keystore/keystore.jks alias: key 2022-10-31 00:53:26,306 INFO EnterpriseSslContextProvider - TLS client using truststore: /secrets/alluxio-tls-truststore/truststore.jks alias: key
TLS termination at Proxy
Users are able to do TLS termination at Proxy pod, that is, the point where the client requests get decrypted.
The Alluxio proxy acts as the server here, so a keystore is required for the proxy. Similar to configure TLS for Alluxio,
a kubernetes secret needs to be created from the keystore file, then configured in the helm chart
.secrets section. For example,
secrets: master: ... worker: ... proxy: client-tls-keystore: client-tls-keystore
Finally, configure the property for Alluxio proxy in the same configuration file, for example,
proxy: properties: alluxio.network.tls.keystore.path: "/secrets/alluxio-tls-keystore/keystore.jks" alluxio.network.tls.keystore.password: "<storepass>" alluxio.network.tls.keystore.key.password: "<keypass>"