Release Notes

2.1.2-1.0

We are excited to announce the release of Alluxio Enterprise 2.1.2-1.0! This is a patch release on the Alluxio Enterprise 2.1.X line. It contains a variety of bug fixes and performance improvements to Alluxio Enterprise 2.1.1-1.0.

Changes

Reduce spammy logs from DynamicResourcePool (27d3c21a3688)
Fix directory index after disk failure (d95e4f458bd3)
Fix Distributed Load Retry Logic (cb3dc8257373)
Fix NPE in GetDatabase (065068187bee)
Add relogin on connect (fe425b7060ef)
Add database information to getDatabase calls (827b9a2e4478)
Fix fingerprints for persisted files (7868b6685055)
Fix the backup restore metrics name (4b0400596ea2)
Add listening address for embedded journal (5952a189eac1)
Fix JobMaster.cancel(jobId) (95a3414dd89b)
Set raft-client policy to connect to leader for journal writes (c6d31fa2e6c0)
Prevent asyncUpdateFileAccessTime from bringing down applications (5ece322dee51)
Add logging to the table syncing process (46dffb208b19)
Implement basic version of getDatabase in catalog service (4f2e78fcda1c)
Keep worker client open after creating data reader/writer (f6ac90479530)
Release clients back into pools during even during context reinit (9fcfcb18ad48)
Add the ability to cache list status results (24a151ebc4f5)
Use DynamicResourcePool for FS and Block clients (f12e3ae2676a)
Shutdown RPC channel immediately if health-check failed (90e9b3b35d73)
Update file fingerprint after UFS migration policy execution
Support configuring netty SSL TLS protocols
Sort string/varchar columns in total order
Use time-based expiration for capability keys

2.1.1-1.0

We are excited to announce the release of Alluxio Enterprise 2.1.1-1.0! This is a patch release on the Alluxio Enterprise 2.1.X line. It contains a variety of bug fixes and performance improvements to Alluxio Enterprise 2.1.0-1.0.

Improvements

Add ability to cache rpc responses (2324e8b4e1e5)
Use 2.2 hive client for greater compatibility (257aa33baa22)
Cleanup file after a failed cp command (c50f3a941229)
Improve trie search (b9b236c6e5d5)
Implement support for terminating cancelled RPCs (a6d9bab1175c)
Process worker metrics asynchronously (28514517d979)
Improve concurrency for MetricsStore (c2d580b12888)
Improve error handling on creating Processes (f80c06ee8652)
Improve error message for Failed Block Reads (661fd62a5f94)
Improve error message on failed mount (8624d6a37536)
Optimize DistributedLoadCommand (c7eb3879f44f)
Remove unnecessary lock to improve async journal writer concurrency (6e32db5c524f)
Parallelize active sync across sync points (9c3e12f3ab3f)
Refine ufs.listStatus granularity for incremental sync (dbd589cf1ca6)
Remove synchronization on mOpenFiles in AlluxioFuseFileSystem (cb6c257c0eb2)
Remove extraneous client RPC to get block info from master (7acf8c2e9afb)
Support session level connection management for leader election (244209fd6d9d)
Reduce the CPU usage of the alluxio master process (097e9a9c5d9f)
Clean up block master service handler (9d0b5583d2e8)
Improve policy scan efficiency
Extract license to values.yaml in Helm template
Hide additional sensitive configuration properties

Issue Fixes

Populate worker webUI fileInfos when blocks are present (8b058cc65a19)
Make rejecting server listen on correct address (9c03e919748e)
Fix traces for RaftJournalWriter (aa4d50c9f676)
Fix NPE in UfsFallbackBlockWriteHandler (ef3ac4688d40)
Load metadata for the path that we are syncing (ad4b1401e03c)
Handle too many attempts properly in DistributedLoadCommand (9c4e3fe1d6eb)
Avoid registering listener for channel state change (7eba7476eb2c)
Log errors instead of throwing exceptions for best-effort send (69dd2fd1749d)
Fix potential NPE on worker start when local mount command fails (730344673c9c)
Fix logging calls with wrong arguments (0e60893c3901)
Fix NPE in UfsDirectoryStatus (512c04b177d6)
Fix race condition building extension properties (273f823c9f35)
Fix relogin for messaging client
Fix error handling for union UFS

2.1.0-1.0

We are extremely excited to announce the release of Alluxio Enterprise Edition 2.1.0-1.0!

This release contains a variety of improvements ranging from user experience, bug fixes, and major performance improvements. A list of changes can be found below.

Highlighted Changes

Alluxio Structured Data Service

This release includes a new Alluxio subsystem for managing and transforming structured data. Structured data comes in the form of databases, tables, and partitions. It is the backbone of many companies’ analytics systems.

Alluxio is entering this space in order to provide performance improvements more than just raw I/O. With this release a simple command can transform data in raw formats (such as CSV) into Parquet files, a much more compact and performant file format which is more suitable for running queries on systems such as Presto. With Alluxio Enterprise Edition, data in tables can be sorted by columns to achieve even higher performance for interactive queries.

Additionally, the release artifacts now include a Presto connector which can be used to connect your Presto cluster to Alluxio.

Kubernetes Helm Chart Deployment

Thanks to community contributions, and a special thanks to the Alibaba Cloud Kubernetes Team a more robust set of Kubernetes templates as well as a helm chart for Kubernetes deployment is now included with Alluxio. Thanks to the team and Alluxio maintainers for this great improvement! Read more about how to use Helm Chart to deploy Alluxio in the Kubernetes environment.

Access Time Based Policy (Enterprise Edition Only)

Alluxio now supports definition data migration policies using access time based condition. Policies defined with this type of condition will be triggered when the last access time of a path is older or newer than a certain period, allowing data management on a per-use basis. Read more about how to set up access time based policies in the documentation.

Transparent URI (Enterprise Edition Only)

Alluxio users often have existing applications accessing their existing storage systems. Previously, users have to change URLs in their applications in order to access the data through Alluxio. In this release, we introduce transparent URI which allows users to access their existing storage systems through Alluxio without changing URIs at application level. Read more about setting up Transparent URI in the documentation.

GCS Version 2 (Enterprise Edition Only)

GCS version 2 is a newer version of Google cloud storage and Alluxio integration using Google Cloud API. It supports using Google application credentials as authentication way instead of interoperability access/secret keypair and has better read/write performance. Read more about setting up GCS version 2 integration in the documentation.

Support for Google Dataproc

A public Google Dataproc init action is now available for users to deploy Alluxio with Google Cloud. Read more about how to deploy Alluxio with Google Dataproc in the documentation.

Reduction of Default Block Size

In this release the default block size in Alluxio has been reduced from 512MB to 64MB. By decreasing the block size, evictions on workers will evict lesser-used data with finer granularity. This can improve use cases where the block size is relatively large compared to the block size of files.

Trial License (Enterprise Edition Only)

Starting from this release, a 30-day trial license is included in the downloaded distribution. The trial license allows users to evaluate Alluxio Enterprise Edition in a test environment during the trial period without having to manually acquire a license.

General Improvements

The embedded journal quorum now utilizes a gRPC-based transport for its RPCs (4095a1c11c)
Process launching can now be done in the foreground (603c6fc291)
Add CLI options for listing last access time (d1d1adffab)
Remove deprecated FaultTolerantFileSystem or alluxio-ft:// (3d7d18dbe2)
Reduce the default block size to 64 MB from 512 MB (9d7338cb58)
Docker containers now handle signals properly (8238b1e2b6)
Hadoop 3.2 is now supported as a UFS (dcfd9cfc1f)
Parallelism option for distributedLoad (f5b70fd71f)
Support for Google Dataproc (ae33402852)
Support for true owner and group with FUSE (a3987c6527)
SCM revision option as a part of alluxio version command (2e67f3be2b)
Removal of MapR support (86e8061ee2)
Blacklist paths for files written with ASYNC_THROUGH (a31ee0a10a)
Upgrade OSS UFS client dependency to 3.6.0 (76aa706215)
Show progress when taking and applying backups: 5d61a3deb5)
Add a command to report policy status
Add a command option to specify a name when adding a policy
Add a configuration property to limit the size of policy action schedule queue
Add a configuration property to turn off policy engine
Automatically sync UFS metadata when performing a policy scan
Improve UFS migration policy nested directory removal efficiency
Add ACL support in Ranger plugin
Remove Kerberos service name property and infer from server principal

Bug Fixes

NPE when UFS journal shutdown fails (f7c8c2e316)
Remove query parameters in when browsing in UI (9a6cc464ed)
NPE on Embedded journal shutdown (cf14aefd6f)
Properly close journal when stopping (fd9b6f942c)
Avoid exception when audit log is used with NOSASL authentication (0e3a152f96)
Sync cache should consider syncing ancestors (b880cef284)
Properly handle interrupt in DynamicResourcePool (ad5c15d6a5)
Properly handle interrupts on various heartbeats (8d2a6ec179)
Properly handle recovering from a journal UFS error (be4e9cb1f4)
Remove duplicate entries in BlockInfo tab of worker WEBUI (bd00866095)
Prevent data loss when writing with ASYNC_THROUGH (b69e73de1e)
Fix union UFS status check APIs
Fix alluxio.underfs.security.authorization.plugin.name property check

Categories