Release Notes

February 28, 2023

• Highlights
  • Default TTL Behavior Change
  • Prevent Recursive Metadata Sync from getStatus Call on a Directory
• Improvements and Bugfixes Since 2.9.1
  • Notable Configuration Property Changes
  • Benchmark
  • Build
  • Client
  • Configuration
  • Docker and K8s
  • Job Service
  • Log
  • Master
  • Metrics
  • S3 API
  • CLI
  • UFS
  • Web UI
  • Worker
• Acknowledgements

Highlights

Default TTL Behavior Change

The default TTL behavior previously was to delete the data from both Alluxio and the underlying storage. This is now changed to only deleting the data from Alluxio, leaving the persisted data untouched. See more details in the corresponding Github issue and PR.

Prevent Recursive Metadata Sync from getStatus Call on a Directory

If a metadata sync is triggered by a getStatus call for inode type NONE, the previous behavior was to trigger metadata sync recursively through the directory’s child nodes. The new behavior does not recursively perform metadata sync, only syncing the target of the getStatus call. If the previous behavior is desired, set the property key alluxio.master.metadata.sync.get.directory.status.skip.loading.children to false.

Improvements and Bugfixes Since 2.9.1

Notable Configuration Property Changes

Benchmark

• Fix the client stressbench concurrency problem (#16934)(cbff62)
• Add local read benchmarks for PagedBlockStore (#16804)(f9cb6d)

Build

• Fix build error when revision is shorter than 8 chars (#16888)(c4b9ec)
• Add maven build dockerfile with jdk17 (#16885)(aea58c)
• Support Maven reproducible builds (#16881)(6400f4)
• Replace ManagementFactoryHelper with ManagementFactory for java11 (#16874)(782b38)

Client

• Remove the restriction of UFS for local cache (#16819)(02bc22)
• Add CapacityBasedDeterministicHashPolicy (#16237)(c14861)
• Implement unbuffer interface for HdfsFileInputStream (#16017)(903269)
• Change default TTL action from DELETE to DELETE_ALLUXIO (#16823)(7aa8b6)
• Reduce useless async cache request when read from remote worker (#16313)(99923c)
• Add cache TTL enforcement (#16843)(6ca7d1)
• Add the two-choice random cache eviction policy (#16828)(530e72)

Configuration

• Support dynamic update HeartbeatThread tick interval (#16702)(7f6baa)
• Fix updated config not notifying config registry (#16764)(407cf1)

Docker and K8s

• Respect MaxRamPercentage in container (#16940)(492cc7)
• Add buildx support for build-docker.sh (#16822)(98dba4)

Job Service

• Fix cli distributed command hang issue (#16786)(59f660)
• Fix logging to capture jobControlId (#16780)(412e54)

Log

• Fix debug log in FileSystemContext#reinit() (#16826)(fb07ea)
• Fix logging to capture jobControlId (#16780)(412e54)

Master

• Improve LS by allowing to omit UFS and mount info (#16893)(3f8c79)
• Fix metadata sync behavior when descendant type is NONE (#16935)(8ef0a4)
• Make standby master keep MountTable up-to-date (#16908)(a5d57e)
• Make workers register to all masters (#16849)(ebeac4)
• Add allowOnStandbyMasters option for version grpc endpoint (#16890)(da6abf)
• Fix file fingerprint to use atomic get content hash of uploaded file (#16597)(00da77)
• Make version service return unavailable on standby masters (#16854)(7399c3)
• Support rm directory and reset DirectChildrenLoadedState (#15647)(778f9e)
• Support gRPC on standby masters (#16839)(325f36)
• Add disable are descendants loaded property (#16813)(5a7cfe)
• Use async journals for block id journal entries in metadata sync (#16529)(9f91f9)

Metrics

• Add metrics sink to job master (#16899)(73f3ce)
• Fix Worker.ActiveClients is negative when load from ufs (#16784)(60d515)
• Add metrics for in memory data structures (#16818)(ee6c23)
• Fix metric worker.cacheBlockSize increases when cache already existed (#16791)(945b10)

S3 API

• Fix incorrect flag passing into delete op (#16941)(a02c5f)
• Add rate limit for s3 proxy (#16866)(f0e9a5)
• Support overwrite option in createFile (#16886)(16ff65)
• Introduce a rearchitectured S3 proxy service (#16654)(f3eca2)

CLI

• Fix head and tail commands read less than expected (#16928)(c4a700)
• Fix free worker command bugs (#16458)(ff0a6d)
• Fix cli distributed command hang issue (#16786)(59f660)

UFS

• Update retry policy for object stores (#16887)(47320b)
• Merge delete requests to Optimize move directory perf on objectstore (#16527)(7ab257)
• Refactor s3 low level output stream and support OSS and OBS (#16122)(0b4796)

Web UI

• Display more information in WebUI Masters (#16636)(8502fb)
• Support display master system status from master UI (#16779)(fb9e88)

Worker

• Support remove blocks on worker for pagestore when free/delete file (#16895)(669f80)
• Fix free worker command bugs (#16458)(ff0a6d)
• Add local read benchmarks for PagedBlockStore (#16804)(f9cb6d)
• Merge local cache invalidation (#16841)(506a0f)
• Split BlockStoreEventListener.onCommitBlock() (#16777)(6fc070)

Acknowledgements

We want to thank the community for their valuable contributions to the Alluxio 2.9.2 release. Especially, we would like to thank:

Deepak Shivamurthy (deepak-shivamurthy), Haoning Sun (Haoning-Sun), Ce Zhang (JySongWithZhangCe), Kaijie Chen (kaijchen), Nandeeshvar Porko Pandiyan (nand-porko), Vimal (vimalKeshu), Xinran Dong(007DXR), Yaolong Liu (codings-dan), Bing Zheng (bzheng888), fanyang (fffanyang), humengyu (humengyu2012), linda (wenfang6), Baolong Mao (maobaolong), Lei Qian (qian0817), Xinyu Deng(voddle), Yangchen Ye (YangchenYe323), yiichan (YichuanSun), and Tianbao Ding (flaming-archer)

Enjoy the new release and look forward to hearing your feedback on our Community Slack Channel.