alluxio.client.file.cache

Class LocalCacheManager

java.lang.Object

alluxio.client.file.cache.LocalCacheManager

All Implemented Interfaces:

CacheManager, CacheStatus, AutoCloseable

@ThreadSafe
public class LocalCacheManager
extends Object
implements CacheManager

A class to manage & serve cached pages. This class coordinates various components to respond for thread-safety and enforce cache replacement policies.

The idea of creating a client-side cache is followed after "Improving In-Memory File System Reading Performance by Fine-Grained User-Space Cache Mechanisms" by Gu et al, which illustrates performance benefits for various read workloads. This class also introduces paging as a caching unit.

Lock hierarchy in this class: All operations must follow this order to operate on pages:

1. Acquire corresponding page lock
2. Acquire metastore lock mMetaLock
3. Update metastore
4. Release metastore lock mMetaLock
5. Update the pagestore and evictor
6. Release corresponding page lock

Nested Class Summary

Nested classes/interfaces inherited from interface alluxio.client.file.cache.CacheManager

CacheManager.Factory, CacheManager.State

Method Summary

Modifier and Type	Method and Description
boolean	append(PageId pageId, int appendAt, byte[] page, CacheContext cacheContext)
void	close()
void	commitFile(String fileId) Commit the File.
static LocalCacheManager	create(CacheManagerOptions options, PageMetaStore pageMetaStore)
boolean	delete(PageId pageId) Deletes a page from the cache.
boolean	delete(PageId pageId, boolean isTemporary) delete the specified page.
void	deleteFile(String fileId) Deletes all pages of the given file.
void	deleteTempFile(String fileId) Deletes all temporary pages of the given file.
int	get(PageId pageId, int pageOffset, int bytesToRead, ReadTargetBuffer buffer, CacheContext cacheContext) Reads a part of a page if the queried page is found in the cache, stores the result in buffer.
int	get(PageId pageId, int pageOffset, ReadTargetBuffer buffer, CacheContext cacheContext) Reads a part of a page if the queried page is found in the cache, stores the result in buffer.
int	getAndLoad(PageId pageId, int pageOffset, int bytesToRead, ReadTargetBuffer buffer, CacheContext cacheContext, java.util.function.Supplier<byte[]> externalDataSupplier) Reads a part of a page if the queried page is found in the cache, stores the result in buffer.
List<PageId>	getCachedPageIdsByFileId(String fileId, long fileLength) Get page ids by the given file id.
Optional<DataFileChannel>	getDataFileChannel(PageId pageId, int pageOffset, int bytesToRead, CacheContext cacheContext) Get a DataFileChannel which wraps a FileRegion.
int	getPageLockId(PageId pageId)
Optional<CacheUsage>	getUsage() Gets cache usage.
void	invalidate(java.util.function.Predicate<PageInfo> predicate) Invalidate the pages that match the given predicate.
boolean	put(PageId pageId, ByteBuffer page, CacheContext cacheContext) Puts a page into the cache manager with scope and quota respected.
CacheManager.State	state()

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface alluxio.client.file.cache.CacheManager

get, get, get, put, put, put

Method Detail

create

public static LocalCacheManager create(CacheManagerOptions options,
                                       PageMetaStore pageMetaStore)
                                throws IOException

Parameters:

options - the options of local cache manager

pageMetaStore - the metadata store for local cache

Returns:

an instance of LocalCacheManager

Throws:

IOException

getDataFileChannel

public Optional<DataFileChannel> getDataFileChannel(PageId pageId,
                                                    int pageOffset,
                                                    int bytesToRead,
                                                    CacheContext cacheContext)
                                             throws PageNotFoundException

Description copied from interface: CacheManager

Get a DataFileChannel which wraps a FileRegion.

Specified by:

getDataFileChannel in interface CacheManager

Parameters:

pageId - the page id

pageOffset - the offset inside the page

bytesToRead - the bytes to read

cacheContext - the cache context

Returns:

an object of DataFileChannel

Throws:

PageNotFoundException

getPageLockId

public int getPageLockId(PageId pageId)

Parameters:

pageId - page identifier

Returns:

the page lock id

put

public boolean put(PageId pageId,
                   ByteBuffer page,
                   CacheContext cacheContext)

Description copied from interface: CacheManager

Puts a page into the cache manager with scope and quota respected. This method is best effort. It is possible that this put operation returns without page written.

Specified by:

put in interface CacheManager

Parameters:

pageId - page identifier

page - page data

cacheContext - cache related context

Returns:

true if the put was successful, false otherwise

commitFile

public void commitFile(String fileId)

Description copied from interface: CacheManager

Commit the File.

Specified by:

commitFile in interface CacheManager

Parameters:

fileId - the file ID

get

public int get(PageId pageId,
               int pageOffset,
               ReadTargetBuffer buffer,
               CacheContext cacheContext)

Description copied from interface: CacheManager

Reads a part of a page if the queried page is found in the cache, stores the result in buffer.

Specified by:

get in interface CacheManager

Parameters:

pageId - page identifier

pageOffset - offset into the page

buffer - destination buffer to write

cacheContext - cache related context

Returns:

number of bytes read, 0 if page is not found, -1 on errors

get

public int get(PageId pageId,
               int pageOffset,
               int bytesToRead,
               ReadTargetBuffer buffer,
               CacheContext cacheContext)

Description copied from interface: CacheManager

Reads a part of a page if the queried page is found in the cache, stores the result in buffer.

Specified by:

get in interface CacheManager

Parameters:

pageId - page identifier

pageOffset - offset into the page

bytesToRead - number of bytes to read in this page

buffer - destination buffer to write

cacheContext - cache related context

Returns:

number of bytes read, 0 if page is not found, -1 on errors

getAndLoad

public int getAndLoad(PageId pageId,
                      int pageOffset,
                      int bytesToRead,
                      ReadTargetBuffer buffer,
                      CacheContext cacheContext,
                      java.util.function.Supplier<byte[]> externalDataSupplier)

Description copied from interface: CacheManager

Reads a part of a page if the queried page is found in the cache, stores the result in buffer. Loads the page otherwise.

Specified by:

getAndLoad in interface CacheManager

Parameters:

pageId - page identifier

pageOffset - offset into the page

bytesToRead - number of bytes to read in this page

buffer - destination buffer to write

cacheContext - cache related context

externalDataSupplier - the external data supplier to read a page

Returns:

number of bytes read, 0 if page is not found, -1 on errors

delete

public boolean delete(PageId pageId,
                      boolean isTemporary)

delete the specified page.

Parameters:

pageId - page identifier

isTemporary - whether is it temporary or not

Returns:

whether the page is deleted successfully or not

delete

public boolean delete(PageId pageId)

Description copied from interface: CacheManager

Deletes a page from the cache.

Specified by:

delete in interface CacheManager

Parameters:

pageId - page identifier

Returns:

true if the page is successfully deleted, false otherwise

state

public CacheManager.State state()

Specified by:

state in interface CacheManager

Returns:

state of this cache

append

public boolean append(PageId pageId,
                      int appendAt,
                      byte[] page,
                      CacheContext cacheContext)

Specified by:

append in interface CacheManager

Returns:

true if append was successful

getCachedPageIdsByFileId

public List<PageId> getCachedPageIdsByFileId(String fileId,
                                             long fileLength)

Description copied from interface: CacheManager

Get page ids by the given file id.

Specified by:

getCachedPageIdsByFileId in interface CacheManager

Parameters:

fileId - file identifier

fileLength - file length (this will not be needed after we have per-file metadata)

Returns:

a list of page ids which belongs to the file

deleteFile

public void deleteFile(String fileId)

Description copied from interface: CacheManager

Deletes all pages of the given file.

Specified by:

deleteFile in interface CacheManager

Parameters:

fileId - the file id of the target file

deleteTempFile

public void deleteTempFile(String fileId)

Description copied from interface: CacheManager

Deletes all temporary pages of the given file.

Specified by:

deleteTempFile in interface CacheManager

Parameters:

fileId - the file id of the target file

invalidate

public void invalidate(java.util.function.Predicate<PageInfo> predicate)

Description copied from interface: CacheManager

Invalidate the pages that match the given predicate.

Specified by:

invalidate in interface CacheManager

close

public void close()
           throws Exception

Specified by:

close in interface AutoCloseable

Throws:

Exception

getUsage

public Optional<CacheUsage> getUsage()

Description copied from interface: CacheStatus

Gets cache usage.

Specified by:

getUsage in interface CacheManager

Specified by:

getUsage in interface CacheStatus

Returns:

cache usage, or none if reporting usage info is not supported