Release Notes

Slack Docker Pulls

2.8.0-2.8

The Alluxio Enterprise 2.8.0-2.8 patch improves worker recovery from potential corruption during process crashes.

Changelog

  • Fix data corrupted issue when laoding from UFS

Configuration changes

None

2.8.0-5.3

The Alluxio Enterprise 2.8.0-5.3 patch includes changes related to the proxy, configuration, and the deadlock fix.

Changelog

  • Fix a deadlock in FileSystemContext (c3ce6aa7e4)
  • Respond with unsupported message when GetBucketLocation (b6f99f6b9d)
  • Add property key to ignore URI authority config in hadoop fs (ea197a5142)

Configuration changes

  • Added alluxio.user.skip.uri.authority with a default value of false

2.8.0-2.7

The Alluxio Enterprise 2.8.0-2.7 patch fixes a potential deadlock scenario when a client requests for the worker list.

Changelog

  • Fix a deadlock in FileSystemContext (c3ce6aa7e4)

2.8.0-5.2

The Alluxio Enterprise 2.8.0-5.2 patch fixes an issue relating to listing multipart uploads via S3 API and distribution of keys to proxies.

Configuration changes

  • Added alluxio.security.authorization.capability.threadpool.size with default value of 1

2.8.0-2.6

The Alluxio Enterprise 2.8.0-2.6 patch includes a change to set the region for the AWS security token service client.

Changelog

  • Migrating Alluxio access from IAM user to IAM role (748e9c1cee)

2.8.0-4.1

The Alluxio Enterprise 2.8.0-4.1 patch includes several previously patched changes related to improving journal snapshots.

Changelog

  • Add compression level option for RocksDB checkpoint (58906307b2)
  • Support multithread checkpointing with compression/decompression (98bf78b771)
  • Change default value for checkpoint compression level (81be291789)
  • Fix snapshot from follower thread leak (8d1703e517)
  • Update raft group only on config change (b6e00e1365)

2.8.0-4.0

The Alluxio Enteprise 2.8.0-4.0 patch includes a fix in S3 API renames.

Changelog

  • Fix incorrect flag passing into delete op (ce9df758d1)
  • Fix FileInfo tokenBag NPE (86618b1513)

2.8.0-1.3

The Alluxio Enterprise 2.8.0-1.3 patch includes RaftClient fix.

Changelog

  • Fix RaftClient closed error (1c7c0683ee)

2.8.0-2.5

The Alluxio Enterprise 2.8.0-2.5 patch includes small fixes for the job service.

Changelog

  • Update job service batch size default value (d8ab2c6b3b)
  • Fix cli distributed command hang issue (ee805bac79)
  • Fix logging to capture jobControlId (7ac9e4b31f)

2.8.0-3.11

The Alluxio Enterprise 2.8.0-3.11 patch includes all the change from 2.8.0-3.10 & 2.8.0-2.4.

Changelog

  • Clean up policy CLI output (377a08023)
  • Add manual trigger command for PDDM (479af42f4e)
  • Remove union fs exception (12904e248)
  • Allows the user to specify the compression level (6e093f078)

    2.8.0-2.4

The Alluxio Enterprise 2.8.0-2.4 patch includes worker error handling.

Changelog

  • Fix null in distributed load cmd output (205be03152)
  • Throw Error after reply error to client (2c07f87e7c)
  • Make worker error propagate to client (4d75d0882c)
  • Add direct mem metrics (ffe6813433)
  • Fix worker swallow OOM (bb73fe3801)

2.8.0-2.3

The Alluxio Enterprise 2.8.0-2.3 patch patch includes a fix for journal checkpointing.

Changelog

  • Fix error in snapshot-taking when using large group ids (4957ce92b3)

2.8.0-2.2

The Alluxio Enterprise 2.8.0-2.2 patch patch includes fixes related to job service and impersonation.

Changelog

  • Fix unwanted message for empty job of distributedLoad (bf45009704)
  • Fix the issue introduced during cherry-pick (722cab1a96)
  • Remove super user for the impersonation in async op (c9014e4755)
  • Add a config item for the persist async temporary directory in hdfs (2276df9994)
  • Add a configurable temp ufs dir (01d71a0b3f)
  • Pass the client user to the thread performing UFS write (2c162508d4)

2.8.0-2.1

The Alluxio Enterprise 2.8.0-2.0 patch includes minor journal backup improvements and additional metrics.

Changelog

  • Add InstrumentedExecutorService for monitoring executor services (8a211c7d3d)
  • Remove duplicate metric for absent cache (cd723791a3)
  • Expose RockDB metrics as Alluxio metrics (f0a32d0a38)
  • Create automatic backup when shutting down Alluxio master (5c39d1ac4b)
  • Add compaction benchmark (7279326aa5)
  • Remove System.exit() calls in ufs journal catchup and propagate errors (ab44195d4d)

2.8.0-2.0

The Alluxio Enterprise 2.8.0-2.0 patch primarily enhances the CLI output of the distributedLoad and distributedCp commands. Two additional fixes related to S3 API uploads and ClientIO stressbench are also included.

Changelog

  • Fix the bug of 2GB File Size Limitation with S3 Rest API (29c0d94fb8)
  • Reformat screen output for sync mode submission and fix wait flag (72f3cf0aa1)
  • Fix client io bench integer overflow (0e624476c2)

2.8.0-1.0

We are thrilled to announce the release of Alluxio Enterprise 2.8.0-1.0! This is the first release on the Alluxio Enterprise 2.8.X line.

Alluxio 2.8 further enhances the S3 API functionality, job service observability, data encryption, and system scalability.

Highlights

Enhanced S3 API with Metadata Tagging

Alluxio 2.8 commences a slew of ongoing improvements to the S3-compatible REST API. Object and bucket tagging APIs have been added (f3d2477) which lay the groundwork for managing end-to-end file metadata through the Alluxio S3 API.

For example the “x-amz-tagging” header is now supported by the PutObject REST endpoint. You can specify user-defined tags to attach to the uploaded object through this request header as a query-parameterized string:

curl -H "x-amz-tagging: key1=value1&key2=value2&flag" -XPUT "http://<host>:<port>/api/v1/s3/<bucket>/<object>" --data="..."

Object tags can be retrieved via:

curl -XGET "http://<host>:<port>/api/v1/s3/<bucket>/<object>?tagging"

See the Alluxio S3 documentation for full details on supported tagging operations. Tags are currently limited to 10 user-defined tags per file in accordance with the S3 specifications.

This release aims to also further compatibility with S3 client applications through request header additions and improvements (e913945).

Stabilize and Scale Alluxio in Deep Training Workloads

Alluxio 2.8 improves the FUSE mount/unmount mechanism. FUSE can be mounted through the Alluxio CLI or via configuration properties (e5f53a9). Fuse unmount mechanism is improved to reduce the chance of leaving Fuse unmounted in the host machines (5393b6). FUSE3 (6f3fe6f) integration is newly supported, enabling future performance and scalability optimizations. Although FUSE2 is the default version, Alluxio will eventually phase out FUSE2 in favor of FUSE3.

Several critical issues were identified in production workloads and fixed to improve the stability of Alluxio when integrating with training workloads. Examples include a RocksDB core crash with high memory consumption (918e73), FUSE segment fault error (issue), and FUSE statfs potential OOM (fabcf47).

More optimizations were added to support a large number of small files and highly concurrent access, including supporting worker registration with millions of blocks (htf8e5e) and improving the performance by 10x and reducing memory overhead when preloading a large number of small files (bc104a9).

Exception Handling for Data Movement

In Alluxio 2.8, we improved the exception handling of data movement, specifically the distributedCp and distributedMv operations. The improvements include removing 0-byte files if the task failed or was canceled (2c83498, 75210e0) and preserving the destination file for overwrites until the write operation succeeds, which previously would have been deleted at the start of the job (2fa2683).

Asynchronous Job Service Execution

Distributed commands for load and copy can now be submitted in asynchronous mode (cfbd06f, 01af5b1). The new mode can be turned on by specifying the --async flag in the distributedCp and distributedMv commands (82505a5).

This new feature simplifies the command submission and monitoring process for the end user. In previous versions, users needed to ensure their command console to remain active and not be interrupted while long-running commands were executing. After launching an asynchronous command, its progress can be reported by running the getCmdStatus CLI command. In the default sync mode, users will no longer see the progress information on files being loaded or copied as they complete. The output will now display the command JobControlId, successfully loaded or copied files, each separated on a new line, statistics of completed and failed files, and the failed file path information.

System Stability and Scalability

In 2.8, we improved the system scalability by fixing concurrency issues with RocksDB (918e73) and resolving leaks (2f2894).

Multiple improvements were made to reduce the general memory usage of Alluxio master. In particular, we improved the performance of recursive deletion and reduced the memory consumption, both by roughly 20%. Based on our performance tests, we have also updated recommendations on how recursive deletion can be done more efficiently. You can find the recommendations in our User-CLI doc.

System Observability

New metrics were added in various parts of the system to provide better observability into the system state, including but not limited to, metadata sync operations (f24c26), data I/O exceptions (250bb3), and distributed commands (696cb8).

Alluxio 2.8 also added the ability for standby masters to serve the Web UI and metrics (a10823).

StressBench Tool Usability

In 2.8 we improved the usability of the StressBench framework by making some parameters more user-friendly. We also added a quick start pipeline to batch run multiple stress tests.

A new workflow has been added under the name MaxFile. The goal of this benchmark is to evaluate the maximum number of files an Alluxio master can handle before becoming unresponsive. The benchmark can run for many hours depending on the metastore type and heap space allocated to the Alluxio master. You can find more information on this document.

Enterprise Edition Highlights

Data Encryption for Enterprise-grade Security

Alluxio 2.8 supports encryption at rest of data in Alluxio managed storage. In conjunction with SSL, this new feature provides full server-side encryption, ensuring end-to-end data security. Data stored on the Alluxio worker is encrypted on write and the data is decrypted when it is read and sent to the client or UFS.

The feature introduces the concept of an encryption zone. Each encryption zone maps to an Alluxio URI, encapsulating files prefixed by the URI. The mapping of paths in the file system enables users to specify which parts of the namespace will be secured by encryption. This provides the ability to balance between security when within an encryption zone and performance for the unspecified paths.

Alluxio creates encryption keys for each encryption zone. The credentials can be stored in Hashicorp Vault or in the master journal. The latter option should only be used for testing or validation.

Alluxio supports different encryption algorithms:

  • AES/CBC/NoPadding and AES/CBC/PKCS5Padding
  • AES/GCM/NoPadding and AES/GCM/PKCS5Padding
  • AES/CTR/NoPadding and AES/CTR/PKCS5Padding
  • AES/ECB/NoPadding and AES/ECB/PKCS5Padding

AES/CTR/NoPadding is the default algorithm.

Read the encryption at rest documentation for more details.

AssumeRole

Alluxio 2.8 introduces a new under storage access token framework. Instead of storing credentials on each worker, workers request for access tokens from the master as needed. These tokens can be permanent or set to expire after a certain duration.

In AWS S3 specific scenarios, users can configure the worker to request for a unique AssumeRole token for either each individual user or for each user and path combination to access S3 objects. The obtained token will expire in an hour and the worker will automatically refresh the token as needed.

When compared to the scenario where each worker independently requests for and manages its own access tokens, the AssumeRole framework can improve the performance by up to 45%.

Read more about configuring the AssumeRole feature for S3 in the documentation.

Policy Driven Data Management Observability

In Alluxio 2.8, we improved the observability of policy driven data management to enable users to diagnose problems when policy initiated jobs fail. A user CLI (eff3d75) has been added and metrics (3a22ce6) are exposed to report the status of the policy engine, policy execution status, and action status. Check out Policy-Driven Data Management for a detailed description of the new CLI.

IMPROVEMENTS AND FIXES

See improvements and fixes since 2.7.0 in their respective release notes:

Enterprise edition changelog

  • Add metrics to policy scan (71ccb79ca2)
  • Add client cert, client key, CA cert and verify configs (e11561b487)
  • Correct the type of vault version (f74953556c)
  • Fix the secure server handler if the encryption is not enabled (efb977bab4)
  • Split encryption meta from block meta (3f555b4f46)
  • Update PDDM Cli for ActionInfo and report (eff3d75aad)
  • Add encryption for tier storage (e815c664db)
  • Fix alluxio.web.login.session.timeout prop key type (52e62fbdbf)
  • Add raft info fsadmin report command (acb5779cd9)
  • Add license check for worker stream register (defd1fa056)
  • Add a config for worker secure rpc port (3bbb9e5a09)
  • Fix the user name in the cache of opa (54179befdb)
  • Modify the capability key to temporary encryption key (3aeafd44bb)
  • Copy ranger-2.1 auth module with generic name (f4fb93cf41)
  • Add counter for assume role related operations (ed315645cd)
  • Add refresh logic for S3 AssumeRole tokens (1afdb11720)
  • Fix the AssumeRole Token generation for Worker S3A UFS instance (26d924875a)
  • Support assume role credential process (d2b4c70af3)
  • Fix space in file name bug (79a8ea68e0)
  • Support the AssumeRole token per resource and operation (fd5ef89086)