Release Notes

Slack Docker Pulls

2.6.0-1.0

We are excited to announce the release of Alluxio Enterprise 2.6.0-1.0! This is the first release on the Alluxio Enterprise 2.6.X line.

Highlights

Unified Alluxio Worker and FUSE Process

Alluxio 2.6 supports running FUSE as a part of a worker process (see docs). Coupling the two helps reduce interprocess communication, which is especially evident in AI/ML workloads where file sizes are small and RPC overheads make up a significant portion of the I/O time. In addition, containing both components in a single process greatly improves the deployability of the software in containerized environments, such as Kubernetes.

Improved Distributed Load

Alluxio 2.6 improves the efficiency of distributed load and its user experience ([documentation])(../operation/User-CLI.html#distributedload), in addition to more traceability and metrics for easier operability. Distributed load command is a key portion commonly used for many users to pre-load data and speed up subsequent training or analytics jobs.

Instrumentation Framework

Alluxio 2.6 adds a large set of metrics and traceability features enabling users to drill into the system’s operating state (documentation). These range from aggregated throughput of the system to summarized metadata latency when serving client requests. The new metrics can be used to measure the current serving capacity of the system and identify potential resource bottlenecks. Request level tracing and timing information can also be obtained for deep performance analysis.

Enterprise Edition Highlights

Improved Security in Job Service

Alluxio 2.6 integrates capabilities with the Job Service, enabling users to securely submit tasks to an Alluxio cluster with capabilities enabled. For more information, see the documentation.

Proxy Support for S3 Authentication

Alluxio 2.6 enables users to authenticate through AssumeRole when using a proxy to connect to S3. Web proxies are useful for providing a single point of inbound and outbound network access, and providing AssumeRole based authentication through a proxy allows for a more secure deployment of Alluxio when using S3 as the underlying storage. Proxy support for S3 can be configured through alluxio.underfs.s3.assumerole.proxy.host and alluxio.underfs.s3.assumerole.proxy.port, see the documentation for more details.

Other Improvements

  • Add and improve metrics
  • Improve log and error message
  • Improve test coverage and code stability
  • Improve Rocks metastore
  • Update dependency version
  • Improve code style and structure
  • Add or improve configurations
  • Improve distributed job commands
  • Improve worker registration
  • Improve kubernetes integration
  • Enhance worker side RPC handling
  • Move container level securityContext to pod level in Kubernetes
  • Improve the metadata sync
  • Support stop master process on demotion
  • Improve journal and add monitor for journal space usage
  • Improve block reader
  • Improve Worker Interface and implementation
  • Implement nondeterministic LRU
  • Support open file for override in Fuse
  • Use Generic type to make CreateFileEntry extensible
  • Add hostAliases in Master and Worker Pods
  • Reduce Mem Copy in client
  • Reduce unnecessary creation of breadcrumbs
  • Improve Web UI
  • Support cephfs direct and native as underfs of alluxio
  • Prevent loading child inode metadata when not required
  • Support cephfs as underfs of alluxio
  • Change default logserver PVC selectors to empty
  • Make connection pools per connection instead of per db for hive udb
  • Support bypass table when attach and sync db

Bug Fixes

  • Fix regression in AbsentUfsPathCache
  • Open the source file before creating target file during migrate
  • Add a boundary check before fuse read
  • Add resource loading fallback logic to ExtensionClassLoader
  • Fix potential directBuffer leak
  • Fix listStatus on open file stream will complete the stream unexpectedly
  • Fix the rejection of legitimate async caching requests because of the fulfill of queue
  • Cancel sync job in middle of ufs calls if client cancelled
  • Fix Fuse write then read problem in async release and support umount
  • Prevent NPE for AbstractFileSystemShellTest
  • Fix helm-chart fuse hostpath type from File to CharDevice
  • Fix master crash due to journal flush error
  • Fix the worker hang issue
  • Add null check for bucket column in Glue UDB
  • Fix alluxio-fuse stat for Alluxio-Worker-FUSE
  • Improve Ceph InputStream to be thread safe
  • Handle client cache failure on “no space left on device”
  • Fix replicate job loading non-persisted files
  • Fix a regression of uncleaned directories
  • Fix process local block in stream
  • Fix packaging distributed load code into DistributedLoadUtils
  • Fix instrumentation setup
  • Fix xstream security issue
  • Fix throw CCE when using Fuse while enable client cache bug
  • Fix possible NullPointerException when calling e.getCause
  • Fix NPE in Allocator and add some UT for Reviewer
  • Fix variable reference and exit status