alluxio.worker.page

Class PagedBlockStore

java.lang.Object

alluxio.worker.page.PagedBlockStore

All Implemented Interfaces:

BlockStore, SessionCleanable, Closeable, AutoCloseable

public class PagedBlockStore
extends Object
implements BlockStore

A paged implementation of LocalBlockStore interface. Implements the block level operations， but instead of using physical block files, we use pages managed by the CacheManager to store the data.

Method Summary

Modifier and Type	Method and Description
void	abortBlock(long sessionId, long blockId) Aborts a temporary block.
void	accessBlock(long sessionId, long blockId) Notifies the block store that a block was accessed so the block store could update accordingly the registered listeners such as evictor and allocator on block access.
void	cleanupSession(long sessionId) Cleans up the session.
void	close()
void	commitBlock(long sessionId, long blockId, boolean pinOnCreate) Commits a block to Alluxio managed space.
static PagedBlockStore	create(UfsManager ufsManager) Create an instance of PagedBlockStore.
String	createBlock(long sessionId, long blockId, int tier, CreateBlockOptions createBlockOptions) Creates a block in Alluxio managed space.
BlockReader	createBlockReader(long sessionId, long blockId, long offset, boolean positionShort, Protocol.OpenUfsBlockOptions options) Creates the block reader to read from Alluxio block or UFS block.
BlockWriter	createBlockWriter(long sessionId, long blockId) Creates a BlockWriter for an existing temporary block which is already created by BlockStore.createBlock(long, long, int, alluxio.worker.block.CreateBlockOptions).
BlockReader	createUfsBlockReader(long sessionId, long blockId, long offset, boolean positionShort, Protocol.OpenUfsBlockOptions options) Creates a block reader to read a UFS block starting from given block offset.
BlockStoreMeta	getBlockStoreMeta() Gets the metadata of the entire store in a snapshot.
BlockStoreMeta	getBlockStoreMetaFull() Similar as BlockStore.getBlockStoreMeta() except that this includes more information about the block store (e.g.
Optional<TempBlockMeta>	getTempBlockMeta(long blockId) Gets the temp metadata of a specific block from local storage.
Optional<BlockMeta>	getVolatileBlockMeta(long blockId) Gets the metadata of a block given its block id or empty if block does not exist.
boolean	hasBlockMeta(long blockId) Checks if the storage has a given block.
boolean	hasTempBlockMeta(long blockId) Checks if the storage has a given temp block.
List<BlockStatus>	load(List<Block> fileBlocks, String tag, OptionalLong bandwidth) Load blocks into alluxio.
void	moveBlock(long sessionId, long blockId, AllocateOptions moveOptions) Moves an existing block to a new location.
OptionalLong	pinBlock(long sessionId, long blockId) Pins the block indicating subsequent access.
void	registerBlockStoreEventListener(BlockStoreEventListener listener) Registers a BlockStoreEventListener to this block store.
void	removeBlock(long sessionId, long blockId) Removes an existing block.
void	removeInaccessibleStorage() Remove Storage directories that are no longer accessible.
void	requestSpace(long sessionId, long blockId, long additionalBytes) Requests to increase the size of a temp block.
void	unpinBlock(long id) Unpins an accessed block based on the id (returned by BlockStore.pinBlock(long, long)).
void	updatePinnedInodes(Set<Long> inodes) Update the pinned inodes.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Method Detail

create

public static PagedBlockStore create(UfsManager ufsManager)

Create an instance of PagedBlockStore.

Parameters:

ufsManager -

Returns:

an instance of PagedBlockStore

pinBlock

public OptionalLong pinBlock(long sessionId,
                             long blockId)

Description copied from interface: BlockStore

Pins the block indicating subsequent access.

Specified by:

pinBlock in interface BlockStore

Parameters:

sessionId - the id of the session to lock this block

blockId - the id of the block to lock

Returns:

a non-negative unique identifier to conveniently unpin the block later, or empty if the block does not exist

unpinBlock

public void unpinBlock(long id)

Description copied from interface: BlockStore

Unpins an accessed block based on the id (returned by BlockStore.pinBlock(long, long)).

Specified by:

unpinBlock in interface BlockStore

Parameters:

id - the id returned by BlockStore.pinBlock(long, long)

commitBlock

public void commitBlock(long sessionId,
                        long blockId,
                        boolean pinOnCreate)
                 throws IOException

Description copied from interface: BlockStore

Commits a block to Alluxio managed space. The block must be temporary. The block will not be persisted or accessible before commitBlock succeeds.

Specified by:

commitBlock in interface BlockStore

Parameters:

sessionId - the id of the client

blockId - the id of the block to commit

pinOnCreate - whether to pin block on create

Throws:

IOException

createBlock

public String createBlock(long sessionId,
                          long blockId,
                          int tier,
                          CreateBlockOptions createBlockOptions)
                   throws WorkerOutOfSpaceException,
                          IOException

Description copied from interface: BlockStore

Creates a block in Alluxio managed space. Calls BlockStore.createBlockWriter(long, long) to get a writer for writing to the block. The block will be temporary until it is committed by BlockStore.commitBlock(long, long, boolean) . Throws an IllegalArgumentException if the location does not belong to tiered storage.

Specified by:

createBlock in interface BlockStore

Parameters:

sessionId - the id of the client

blockId - the id of the block to create

tier - the tier to place the new block in BlockStoreLocation.ANY_TIER for any tier

createBlockOptions - the createBlockOptions

Returns:

a string representing the path to the local file

Throws:

WorkerOutOfSpaceException - if this Store has no more space than the initialBlockSize

IOException

createBlockReader

public BlockReader createBlockReader(long sessionId,
                                     long blockId,
                                     long offset,
                                     boolean positionShort,
                                     Protocol.OpenUfsBlockOptions options)
                              throws IOException

Description copied from interface: BlockStore

Creates the block reader to read from Alluxio block or UFS block. Owner of this block reader must close it or lock will leak.

Specified by:

createBlockReader in interface BlockStore

Parameters:

sessionId - the client session ID

blockId - the ID of the UFS block to read

offset - the offset within the block

positionShort - whether the operation is using positioned read to a small buffer size

options - the options

Returns:

a block reader to read data from

Throws:

IOException - if it fails to get block reader

createUfsBlockReader

public BlockReader createUfsBlockReader(long sessionId,
                                        long blockId,
                                        long offset,
                                        boolean positionShort,
                                        Protocol.OpenUfsBlockOptions options)
                                 throws IOException

Description copied from interface: BlockStore

Creates a block reader to read a UFS block starting from given block offset. Owner of this block reader must close it to cleanup state.

Specified by:

createUfsBlockReader in interface BlockStore

Parameters:

sessionId - the client session ID

blockId - the ID of the UFS block to read

offset - the offset within the block

positionShort - whether the operation is using positioned read to a small buffer size

options - the options

Returns:

the block reader instance

Throws:

IOException - if it fails to get block reader

abortBlock

public void abortBlock(long sessionId,
                       long blockId)
                throws IOException

Description copied from interface: BlockStore

Aborts a temporary block. The metadata of this block will not be added, its data will be deleted and the space will be reclaimed. Since a temp block is "private" to the writer, this requires no previously acquired lock.

Specified by:

abortBlock in interface BlockStore

Parameters:

sessionId - the id of the session

blockId - the id of a temp block

Throws:

IOException

requestSpace

public void requestSpace(long sessionId,
                         long blockId,
                         long additionalBytes)
                  throws WorkerOutOfSpaceException,
                         IOException

Description copied from interface: BlockStore

Requests to increase the size of a temp block. Since a temp block is "private" to the writer client, this operation requires no previously acquired lock.

Specified by:

requestSpace in interface BlockStore

Parameters:

sessionId - the id of the session to request space

blockId - the id of the temp block

additionalBytes - the amount of more space to request in bytes, never be less than 0

Throws:

WorkerOutOfSpaceException - if requested space can not be satisfied

IOException

load

public List<BlockStatus> load(List<Block> fileBlocks,
                              String tag,
                              OptionalLong bandwidth)

Description copied from interface: BlockStore

Load blocks into alluxio.

Specified by:

load in interface BlockStore

Parameters:

fileBlocks - list of fileBlocks, one file blocks contains blocks belong to one file

tag - the user/client name or specific identifier

bandwidth - limited bandwidth to ufs

Returns:

load status for failed blocks

createBlockWriter

public BlockWriter createBlockWriter(long sessionId,
                                     long blockId)
                              throws IOException

Description copied from interface: BlockStore

Creates a BlockWriter for an existing temporary block which is already created by BlockStore.createBlock(long, long, int, alluxio.worker.block.CreateBlockOptions).

Specified by:

createBlockWriter in interface BlockStore

Parameters:

sessionId - the id of the client

blockId - the id of the block to be opened for writing

Returns:

the block writer for the local block file

Throws:

IOException

moveBlock

public void moveBlock(long sessionId,
                      long blockId,
                      AllocateOptions moveOptions)
               throws WorkerOutOfSpaceException,
                      IOException

Description copied from interface: BlockStore

Moves an existing block to a new location.

Specified by:

moveBlock in interface BlockStore

Parameters:

sessionId - the id of the session to move a block

blockId - the id of an existing block

moveOptions - the options for move

Throws:

WorkerOutOfSpaceException - if newLocation does not have enough extra space to hold the block

IOException

removeBlock

public void removeBlock(long sessionId,
                        long blockId)
                 throws IOException

Description copied from interface: BlockStore

Removes an existing block. If the block can not be found in this store.

Specified by:

removeBlock in interface BlockStore

Parameters:

sessionId - the id of the session to remove a block

blockId - the id of an existing block

Throws:

IOException

accessBlock

public void accessBlock(long sessionId,
                        long blockId)

Description copied from interface: BlockStore

Notifies the block store that a block was accessed so the block store could update accordingly the registered listeners such as evictor and allocator on block access. //TODO(beinan): looks like we should not expose this method except the test

Specified by:

accessBlock in interface BlockStore

Parameters:

sessionId - the id of the session to access a block

blockId - the id of an accessed block

getBlockStoreMeta

public BlockStoreMeta getBlockStoreMeta()

Description copied from interface: BlockStore

Gets the metadata of the entire store in a snapshot. There is no guarantee the state will be consistent with the snapshot after this method is called. This function should be cheap since it is called for every block.

Specified by:

getBlockStoreMeta in interface BlockStore

Returns:

store metadata

getBlockStoreMetaFull

public BlockStoreMeta getBlockStoreMetaFull()

Description copied from interface: BlockStore

Similar as BlockStore.getBlockStoreMeta() except that this includes more information about the block store (e.g. blockId list). This is an expensive operation.

Specified by:

getBlockStoreMetaFull in interface BlockStore

Returns:

full store metadata

getTempBlockMeta

public Optional<TempBlockMeta> getTempBlockMeta(long blockId)

Description copied from interface: BlockStore

Gets the temp metadata of a specific block from local storage.

Specified by:

getTempBlockMeta in interface BlockStore

Parameters:

blockId - the id of the block

Returns:

metadata of the block if the temp block exists

hasBlockMeta

public boolean hasBlockMeta(long blockId)

Description copied from interface: BlockStore

Checks if the storage has a given block.

Specified by:

hasBlockMeta in interface BlockStore

Parameters:

blockId - the block id

Returns:

true if the block is contained, false otherwise

hasTempBlockMeta

public boolean hasTempBlockMeta(long blockId)

Description copied from interface: BlockStore

Checks if the storage has a given temp block.

Specified by:

hasTempBlockMeta in interface BlockStore

Parameters:

blockId - the temp block id

Returns:

true if the block is contained, false otherwise

getVolatileBlockMeta

public Optional<BlockMeta> getVolatileBlockMeta(long blockId)

Description copied from interface: BlockStore

Gets the metadata of a block given its block id or empty if block does not exist. This method does not require a lock id so the block is possible to be moved or removed after it returns.

Specified by:

getVolatileBlockMeta in interface BlockStore

Parameters:

blockId - the block id

Returns:

metadata of the block

cleanupSession

public void cleanupSession(long sessionId)

Description copied from interface: SessionCleanable

Cleans up the session.

Specified by:

cleanupSession in interface SessionCleanable

Parameters:

sessionId - the session id

registerBlockStoreEventListener

public void registerBlockStoreEventListener(BlockStoreEventListener listener)

Description copied from interface: BlockStore

Registers a BlockStoreEventListener to this block store.

Specified by:

registerBlockStoreEventListener in interface BlockStore

Parameters:

listener - the listener to those events

updatePinnedInodes

public void updatePinnedInodes(Set<Long> inodes)

Description copied from interface: BlockStore

Update the pinned inodes.

Specified by:

updatePinnedInodes in interface BlockStore

Parameters:

inodes - a set of inodes that are currently pinned

removeInaccessibleStorage

public void removeInaccessibleStorage()

Description copied from interface: BlockStore

Remove Storage directories that are no longer accessible.

Specified by:

removeInaccessibleStorage in interface BlockStore

close

public void close()
           throws IOException

Specified by:

close in interface Closeable

Specified by:

close in interface AutoCloseable

Throws:

IOException