Release Notes

2.10.0-3.4

The Alluxio Enterprise 2.10.0-3.4 patch improves batch delete operations in object stores like S3.

Changelog

• Handle batch delete failures with retries

Configuration changes

Property keys added

• alluxio.underfs.delete.retry.max.num=10

2.10.0-3.3

The Alluxio Enterprise 2.10.0-3.3 patch fixes the checksum command and adds an endpoint to refresh master keys on workers.

Changelog

• Release BlockReader in CRC check
• Add restful API to refresh capability keys

Configuration changes

None

2.10.0-3.2

The Alluxio Enterprise 2.10.0-3.2 patch adds logging configurations for diagnosing large metadata operations and adds a checksum command to verify data integrity.

Changelog

• Add more extensive logs and skip expensive steps in large metadata operations
• Read S3 API exception when read size is much larger than the range size
• Add checksum command

Configuration changes

Property keys added

• alluxio.master.metrics.rm.size.distribution.buckets=1000,10000,100000,1000000,10000000
• alluxio.master.metrics.ls.size.distribution.buckets=1000,10000,100000,1000000,10000000
• alluxio.master.file.delete.unchecked=false
• alluxio.master.file.delete.skip.sync=false
• alluxio.master.large.operation.warning.threshold=1000000
• alluxio.user.file.delete.skip.sync=false
• alluxio.worker.block.checksum.calculation.throughput.threshold=-1
• alluxio.worker.block.checksum.calculation.thread.pool.size=16
• alluxio.worker.block.checksum.calculation.use.buffer.pool=false
• alluxio.useer.client.checksum.calculation.batch.size=2147483647

2.10.0-3.1

The Alluxio Enterprise 2.10.0-3.1 patch fixes ranged copy related operations in the S3 API.

Changelog

• Support range copy in copyPart in the way of through_cache
• Avoid the NPE exception in copy object with x-amz-tagging-directive
• Support range copy in copyObject and copyPart

Configuration changes

None

2.10.0-3.0

The Alluxio Enterprise 2.10.0-3.0 patch allows the tracking of vCPU utilization among the cluster processes. Users upgrading from a prior version must obtain a new license to be compatible with the new license enforcement in 2.10.0-3.0.

Changelog

• Add licensing to vCPU utilization
• Rename alluxio.underfs.s3.threads.max to alluxio.underfs.s3.connections.max
• Fix 2.10 replaying a 2.8 journal

Configuration changes

Property keys renamed

• alluxio.underfs.s3.threads.max renamed to alluxio.underfs.s3.connections.max. Note that the previous key will continue to work.

2.10.0-2.15

The Alluxio Enterprise 2.10.0-2.15 patch adds a retention time for submitted jobs, avoiding a potential OOM for the job master after handling a large number of jobs.

Changelog

• Fix job-master leak memory when submitting distributed jobs

Configuration changes

Property keys added and their default values

• alluxio.job.master.job.trace.retention.time=1d

2.10.0-2.14

The Alluxio Enterprise 2.10.0-2.14 patch applies 2 bugfixes.

Changelog

• Downgrade AWS SDK to 2.15.79 for netty compatibility
• Disable invalidation time to fix a potential cache eviction recursion issue

Configuration changes

None

2.10.0-2.13

The Alluxio Enterprise 2.10.0-2.13 patch adds support for delegation token when using transparent URI.

Changelog

• Support building token uri in ShimFS

Configuration changes

None

2.10.0-2.12

The Alluxio Enterprise 2.10.0-2.12 patch fixes a file permissions bug when in getFileInfo when using Ranger policies.

Changelog

• Replace read with execute permission check in getFileInfo method

Configuration changes

None

2.10.0-2.11

The Alluxio Enterprise 2.10.0-2.11 patch fixes a bug where block readers may not be closed when exceptions are thrown. This bug was introduced in 2.10.0-2.6, so it is highly recommended to upgrade to this patch if using 2.10.0-2.6 or later.

Changelog

• Close reader when exception happens

Configuration changes

None

2.10.0-2.10

The Alluxio Enterprise 2.10.0-2.10 patch adds a rate limiter feature for UFS reads and improves the COS UFS.

Changelog

• Change the timing of acquire permits to when opening UFS range stream
• Support range read api in cos ufs
• Support UFS ratelimiter
• Return 404 for HEAD requests with files ending with slash

Configuration changes

Property keys added and their default values

• alluxio.worker.data.ufs.read.rate.limit.mb=0

2.10.0-2.9

The Alluxio Enterprise 2.10.0-2.9 patch enhances the distributed load feature and improves proxy endpoints for S3 API.

Changelog

• Support exposing 503 SlowDown error code to Proxy responses
• Skip the idempotency check in UFS
• Fix memory leaking when loading data
• Fix status of failed jobs
• Add batch size for load
• Add configurable job failure parameters
• Support reading NO_CACHE file data from UFS directly in proxy
• Add position read supports for s3 proxy
• Add a switch to bypass the unnecessary prefix /api/v1/s3 in Proxy V2
• Fix the metadata error in first load

Configuration changes

Property keys added and their default values

• alluxio.job.load.failure.count.threshold=100
• alluxio.job.load.failure.ratio.threshold=0.05
• alluxio.proxy.s3.use.position.read.range.size=0
• alluxio.proxy.s3.api.nocache.ufs.read.through.enabled=false
• alluxio.proxy.s3.api.noprefix.enabled=false

2.10.0-2.8

The Alluxio Enterprise 2.10.0-2.8 patch adds rate limiting support to the S3 API and updates several metrics. The CLI commands for fs distributedMv and fs distributedLoad are removed; use the distributed version of the fs cp and fs load instead.

Changelog

• Fix cacheMissPercentage metric
• Remove noisy logging in InstanceMetadataServiceResourceFetcher
• Fix the active rpc metrics
• Fix the issue list request can not be limited by s3 rate limiter
• Support request throttle in S3 API for different users
• Change proxy register port
• Remove distributedLoad
• Remove distributedMv

Configuration changes

Property keys added and their default values

• alluxio.proxy.s3.quota.user.{username}.{get

  put

  list}.request.max=0

• alluxio.proxy.s3.throttle.max.wait.time.ms=60000

2.10.0-2.7

The Alluxio Enterprise 2.10.0-2.7 patch improves asynchronous processing for the S3 API and shades additional netty files.

Changelog

• Shade ALPN libraries
• Add global ratelimiter and make async timeout configurable in Proxy V2
• Remove s3 handler map in s3 proxy

Configuration changes

Property keys added and their default values

• alluxio.proxy.s3.v2.async.context.timeout.ms=30000

2.10.0-2.6

The Alluxio Enterprise 2.10.0-2.6 patch improves worker behavior with respect to pinned files and recovery from potential corruption during process crashes.

Changelog

• Set min replication when TTL expires to free a file
• Fix potential code injection vulnerability
• Fix data corrupted issue when loading from UFS

Configuration changes

None

2.10.0-2.5

The Alluxio Enterprise 2.10.0-2.5 patch adds security configurations, improves metadata sync throttling, and tracks Netty memory usage.

Changelog

• Add a config to distinguish the tls configs for data and HA journal
• Improve metadata sync throttling
• Add the metrics to record the direct memory used by Netty
• Specify explicit the server uri if delegation token is enabled

Configuration changes

Property keys added and their default values

• alluxio.network.journal.tls.enabled=true
• alluxio.network.tls.use.system.trusstore= false
• alluxio.network.tls.client.no.endpoint.identification=false
• alluxio.security.authentication.delegation.token.server.uri.match=””

2.10.0-2.4

The Alluxio Enterprise 2.10.0-2.4 patch adds a whitelist for transparent URI and force bypass filesystem to use s3a instead of deprecated s3 or s3n clients.

Changelog

• Fix configuration for shimfs to use a non-default config
• Add allowlist (whitelist) to transparent uri
• Add Regex support to direct access
• Use s3a for all s3 compatible scheme when bypass is used
• Add Insecure ssl

Configuration changes

Property keys added and their default values

• alluxio.user.shimfs.allow.list=””

2.10.0-2.3

The Alluxio Enterprise 2.10.0-2.3 patch includes configurable improvements to UFS read and metadata sync behaviors as well as improvements to the S3 API. It is recommended to set the newly added property alluxio.underfs.read.chunk.size to higher value, such as 4MB, for improved UFS read performance. If a user has an object store as UFS and does not rely on inode permissions in Alluxio for access control, setting alluxio.master.metadata.sync.recursive.load.parent.dirs=false will improve metadata sync performance. This is especially helpful when the UFS does not have a definition of directory, such as Alibaba OSS.

Changelog

• Fix ufs read timeout
• Add an option to skip persist/check parent dirs in metadata sync
• Fix monitor process if delegation token enabled
• Respond with unsupported message when GetBucketLocation
• Load content hash as the etag of the object when the UFS is S3

Configuration changes

Property keys added and their default values

• alluxio.master.metadata.sync.recursive.load.parent.dirs=true
• alluxio.underfs.read.chunk.size=8KB

2.10.0-2.2

The Alluxio Enterprise 2.10.0-2.2 patch includes improvements to file permissions and a feature to avoid accessing the cache for paths specified in configuration.

Changelog

• Add direct access option to avoid caching certain paths
• Get User/Group/Permission of existed objects in OSS ufs
• Support Mkdir/CreateFile with configured default umask in HDFS API
• Add worker vCPU info to fsadmin report capacity command

Configuration changes

Property keys added and their default values

• alluxio.underfs.oss.default.mode=0700
• alluxio.underfs.oss.owner.id.to.username.mapping=””
• alluxio.user.file.direct.access=””

Property keys removed

None

Property keys with modified default values

None

2.10.0-2.1

The Alluxio Enterprise 2.10.0-2.1 patch includes fixes in the areas of the proxy service, S3 API multipart uploads, and pinned files.

Several new capabilities can be enabled via configuration or flags:

• Add an option to deny permissions in the case where the requested path is not defined by any Ranger policy
• Support configuring multiple independent KDCs

For the CLI commands bin/alluxio fs ls and bin/alluxio fs loadMetadata, there will be a limit of 1 million files imposed on the operation. To ignore this limit, add the -n flag. This restores the previous behavior but is not recommended. To set a different limit, set -Dalluxio.user.file.metadata.sync.limit=<newLimit> when executing the bin/alluxio command.

Changelog

• Avoid copying properties in HdfsUnderFileSystem
• Alluxio talks to multiple independent KDCs in the same cluster.
• Enhance Ranger policy by Implicit deny for requests to paths not covered by Ranger ACL policies
• Fix key distribution for proxy
• Replication Checker shall remove pinned file if not found in UFS
• Alluxio S3 API “list-multipart-uploads” returns no results with “multipart.upload.write.through” enabled
• Trino client stuck on acquiring a lock for getting the worker lists from the master node
• Add limits to ls -R operations

Configuration changes

Property keys added and their default values

• alluxio.security.authorization.capability.threadpool.size=1
• alluxio.security.authorization.default.deny=false
• alluxio.security.kerberos.min.seconds.before.relogin=1min
• alluxio.security.underfs.hdfs.kerberos.login.per.instance=false
• alluxio.security.user.information.executor.min.thread.number=1
• alluxio.security.user.information.executor.schedule.interval=1min
• alluxio.user.file.metadata.sync.limit=1000000

Property keys removed

• alluxio.master.persistence.checker.interval
• alluxio.master.persistence.initial.interval
• alluxio.master.persistence.max.interval
• alluxio.master.persistence.max.total.wait.time
• alluxio.master.persistence.scheduler.interval
• alluxio.user.file.replication.durable
• alluxio.user.file.ufs.tier.enabled

Property keys with modified default values

None

2.10.0-2.0

The Alluxio Enterprise 2.10.0-2.0 patch introduces bug fixes and improvements in the areas of metadata sync and graceful shutdown of worker process.

Changelog

• Fix a bug that creates 0 byte block file mistakenly
• Fix corrupted block causing reads to fail
• Fix s3 force path style
• Implement metadata sync V2
• Fix FileSystemMergeJournalContext related
• Make capacity command show worker state
• Support gracefully shutdown worker
• Improve proxy register functionality and command

2.10.0-1.0

We are thrilled to announce the release of Alluxio 2.10.0! This is the first release on the Alluxio 2.10.X line. This release introduces significant improvements in handling journal snapshots to reduce the duration of master failovers. Multiple improvements and fixes were also made for the S3 API, metadata sync, as well as the Kubernetes helm chart and operator.

Highlights

Journal snapshot

The embedded journal relies on the snapshot mechanism to keep each master’s journal from growing unbounded and to enable fast recovery on failovers and crashes. The implementation has been modified to increase generation and recovery performance by 95%. These changes will greatly contribute to decreasing the duration of master unresponsiveness in the case of a failover event. See the pull request for more details.

Proxy server refactor for improved S3 API

The proxy server now offers a beefier request handling framework, separating high vs low latency requests to resolve the noisy neighbor problem in flow control. This refactor of the proxy server results in better control over various scenarios to allow Alluxio to respond more accurately according to the S3 API specification. By decoupling the dependence on Jetty, there is more control on handling the I/O from ingress and egress through the HTTP connection. The new version of the proxy server can be enabled by setting the property value alluxio.proxy.s3.v2.version.enabled to be true. See the pull request for more information.

[EE] S3 API OIDC authentication integration with OPA

The proxy server will pass the OIDC token (JWT token) to OPA if the user sends the token as part of the S3 API request and OPA authorization is enabled in Alluxio. The user then can add logic in their OPA policy to utilize the JWT token to interface with authorization providers external to Alluxio. This feature enables the user to add or remove any fields in the JWT token without modifying Alluxio and validate the fields independently. See the documentation for more information.

[EE] K8s helm chart and operator enhancements

Different sets of configuration files for deploying Alluxio on Kubernetes are provided with helm charts. With the new sets of default configurations, the Alluxio cluster has better scalability and performance on certain analytical queries. Configurable pod status check is now enabled on the Kubernetes operator. Using built-in probes, Kubernetes will periodically check the health of the pods in the cluster.

Improvements and fixes

Notable configuration property updates

Changelog since 2.9.3

Refer to previous release notes for recent open source releases:

• 2.9.3
• 2.9.2
• 2.9.1

Benchmark

• Fix stress create tree file number (#17136) (88c88b)

Journal and Metastore

• Fix Rocksdb thread safety using refcount(no lock) (#17171) (9f152c)
• Enhance embedded journal checkpointing significantly (#16998) (8cbcbc)

Master

• Support cron timer to arrange the period heartbeat executor invoke time (#16900) (fd19fb)
• Avoid getting meta sync lock if no need (#17172) (875f3f)

S3 API and Proxy

• Enable bucket cache in v2 s3 proxy (#17022) (46270b)
• Fix CopyObject writetype and unclosed outstream in InitiateMPUpload (#17164) (01bfec)

CLI

• Skip ssh for localhost (#17167) (5f1704)
• Show master versions in fsadmin report command (#17177) (9a4e15)
• Support executing runTests on specific workers (#17170) (aee3c5)
• Fix the issue where runTest ignores arguments (#17068) (db9a76)

UFS

• Check filesystem for mount point (#17159) (a600f1)

Worker

• Add worker decommission step 1 (#14000) (50c4c2)

Enterprise changelog since 2.9.0-2.0

Refer to previous release notes for recent enterprise releases:

• 2.9.0-2.0

Changelog

• Add distributed copy command line (37418f61da)
• Fix shading for ByteBuddy (8d388061c5)
• Add region for AWS security token service client (5a33b7714b)
• Handle more methods from FSDataInputStream in ShimHdfsDataInputStream (7b348fcb35)
• Fix taking wrong metrics in collectUtilization command (e2fc04f916)
• Add collectUtilization command (c967539c57)
• Fix FileInfo tokenBag NPE (8cba187570)
• Add path based policy for PDDM (d9c92d348f)
• Turn off merge inode journals property (a7b8ae3133)