The Alluxio Enterprise 2.9.0-2.0 patch includes many fixes in the areas of master stability, metadata sync, S3 API, and job service.
- Dump more information on Alluxio process crash/master failover
- Add compression level option for RocksDB checkpoint
- Change default value for checkpoint compression level
- Improve state lock tracking and report on error
- getStatus will list all metadata in UFS
- Update raft group only on config change
- Fix concurrent sync dedup
- S3 API metadata sync on-demand
- Fix incorrect flag passing into delete op in multipart uploads
- 404 error with multi-part uploads via Alluxio S3 API
- Add encoding-type support for S3 ListObjects
Job service and PDDM
- Add ability to trigger PDDM policy manually
- Update default value for job service batch size
- Job hang and failed because the task is wrongly judged as failed by the job master
- Long running job tried to cancel but job did not get cancelled
- Remove union fs exception
- Clean up policy CLI output
- Fix NPE in distributed cmd
We are thrilled to announce the release of Alluxio 2.9.0! This is the first release on the Alluxio 2.9.X line. This release introduces a feature for fine-grained caching of data, metrics for monitoring the master process health, changes to the default configuration to better handle master failover and journal backups. Multiple improvements and fixes were also made for the S3 API, helm charts, and POSIX API.
Paging storage on Workers
The Alluxio workers support fine-grained page-level caching, typically at the 1 MB size, as an alternative to the existing block-based tiered caching, which defaults to 64 MB. Through this feature, caching performance will be improved by reducing amplification of data read by applications. See the documentation for more details.
Master monitoring metrics
The Alluxio master periodically checks its resource usage,
including CPU and memory usage, and several internal data structures that are performance critical.
By inspecting snapshots of resource utilization metrics, the state of the system can be inferred,
which can be retrieved by inspecting the
master.system.status metric. The possible statuses are:
The monitoring indicators describe the system status in a heuristic way to have a basic understanding of its load. See the documentation for more information about monitoring.
Journal and failover stability
The default configuration as of 2.9.0 skips the block integrity check upon master startup and failover (a493b69e2d).
This speeds up failovers considerably to minimize system downtime during master leadership transfers.
Instead, block integrity checks will be performed in the background periodically so as not to interfere with normal master operations.
Another default configuration change will delegate the journal backup operation to a standby master (e3ed7b674f)
so as to not block the leading master’s operations for an extended period of time.
--allow-leader flag to allow the leading master to also take a backup
or force the leader to take a backup with the
See the documentation for additional information about backup delegation.
Enterprise Edition Highlights
Cross cluster synchronization
For use cases where multiple Alluxio clusters mount the same UFS path and writes go through Alluxio, the cross cluster synchronization feature can be enabled, ensuring that the files under the mount points are consistent across clusters. When a path is modified in Alluxio a modification notification will be sent to all other clusters that also mount this path. The next time this path is accessed a metadata sync will be performed with the UFS updating the metadata on the cluster to the latest version.
Prior to 2.9.0 this could be achieved through the polling based metadata sync feature where metadata is synchronized based on a user defined interval. By using cross cluster synchronization instead, the load on both Alluxio and the UFS will be reduced as metadata is only synced on paths that have been modified when accessed. Read more about cross cluster synchronization in the documentation.
Alluxio Kubernetes Operator
The Alluxio Operator is an operator for managing an Alluxio cluster inside a Kubernetes cluster. It manages the full lifecycle, from deployment to maintenance, and also includes automating upgrades. With the Alluxio Operator, users no longer need to configure and maintain a large amount of configurations.
The cluster specification and configuration is defined by writing a Custom Resource. The Alluxio Operator detects and analyzes the configurations in the submitted Custom Resource to either create or update the Alluxio cluster. Learn more about the Alluxio Operator in the documentation.
Authentication via S3 API
Alluxio adapts the OIDC protocol to handle third-party authentication when interfacing with Alluxio through its S3 API. By verifying the JWT token provided from the OIDC provider, user identities will be verified before their requests are processed.
This feature enables the customer to connect to identity management systems to leverage single sign on (SSO). Alluxio supports integration with several OIDC providers, including PingFederate and Amazon Cognito. See the documentation for more information.
Encryption at rest performance optimization
Alluxio introduces the encryption by chunk algorithm for encrypting stored data and also supports the use of JCE or OpenSSL libraries to en/decrypt. These new methods improve the overall performance for encrypting data at rest.
Based on these recommendations with respect to performance and security level,
the default encryption method and cipher type are changed to
Using a SQL workload as a benchmark, the new default settings decrease the performance overhead of encryption
from 400% to 120% when compared to running without encryption enabled.
IMPROVEMENTS AND FIXES
See improvements and fixes since 2.8.0 in their respective release notes:
Notable configuration property updates
| Property key | Old 2.8.0-1.0 value | New 2.9.0-1.0 values |
alluxio.fuse.auth.policy.class | alluxio.fuse.auth.SystemUserGroupAuthPolicy | alluxio.fuse.auth.LaunchUserGroupAuthPolicy |
alluxio.underfs.s3.assumerole.session.scope | USER_PATH | USER |
alluxio.master.metrics.heap.enabled | true | false |
alluxio.master.periodic.block.integrity.check.repair | false | true |
alluxio.master.startup.block.integrity.check.enabled | true | false |
alluxio.security.tier.storage.encryption.cipher.type | AES/CTR/NoPadding | AES/GCM/NoPadding |
alluxio.user.client.cache.async.write.enabled | true | false |
Enterprise edition changelog
- Add retention time for job (8aa790abc2)
- Avoid double counted failed files in load progress (2cfe62bfb4)
- Add logging and metrics for load (b3ec7a382a)
- Fix file iterator for load manager (5f51a084d2)
- Add missing option to new LoadCommand (bf55611b64)
- Support automatically adjust load batch size (38f641e8b7)
- Support load progress report options in cli (c6dbd9866d)
- Support using partial listing in load (cef9fa84c9)
- Support JSON format for load progress report (bdddbd17c8)
- Fix stats tracking in LoadJob (bb93616278)
- Add support for getting load progress (5689cce1ab)
- Support authentication in LoadManager (fea1dd3cc5)
- Support submitLoad and stopLoad in alluxio client (f21a377453)
- Implement Journaled API for LoadManager (7933b70389)
- Bring back LoadManager (f8f8ecd9b7)
- Modify the default assume role scope to USER (a90d405ddb)
- Remove super user for the impersonation in async op (7e6200d8ef)
- Fix the behavior when the ranger is enabled (74b022b634)
- Add a config item for the hashicorp KV store prefix (27181b99d3)
- Add KMIP connector to connect a KMIP backed KMS for encryption (1b283775b9)
- Pass the client user to the thread performing UFS write (454cab2418)
- Add kmip plugin code (981c7df923)
- Add assume role application backoff retry logic (af38f9efd2)
- Add properties for kmip client (b2771ee22b)
- Add openssl lib software link in docker container (79c1acd7bd)
- Add openssl and jce support for encryption (ec1dc90740)
S3 API and Proxy
- Enable TLS in Alluxio on k8s (24071b61e9)
- Fix S3 Proxy TLS without an alias set (d06c6e31ae)
- Enable TLS S3 Proxy with new config items (2e48d42264)
- Enable the secure web server for s3 proxy webserver (c6c53b11cd)
- Remove privileges (809d792096)
- Remove entry point for OSS, WEB, SWIFT, CEPH, OZONE, COS, KODO (1245de70ec)