alluxio.worker.block

Interface LocalBlockStore

All Superinterfaces:

AutoCloseable, Closeable, SessionCleanable

All Known Implementing Classes:

TieredBlockStore

public interface LocalBlockStore
extends SessionCleanable, Closeable

A blob store interface to represent the local storage managing and serving all the blocks in the local storage.

Method Summary

Modifier and Type	Method and Description
void	abortBlock(long sessionId, long blockId) Aborts a temporary block.
void	accessBlock(long sessionId, long blockId) Notifies the block store that a block was accessed so the block store could update accordingly the registered listeners such as evictor and allocator on block access.
void	cleanupSession(long sessionId) Cleans up the data associated with a specific session (typically a dead session).
void	commitBlock(long sessionId, long blockId, boolean pinOnCreate) Commits a temporary block to the local store.
BlockLock	commitBlockLocked(long sessionId, long blockId, boolean pinOnCreate) Similar to commitBlock(long, long, boolean).
TempBlockMeta	createBlock(long sessionId, long blockId, AllocateOptions options) Creates the metadata of a new block and assigns a temporary path (e.g., a subdir of the final location named after session id) to store its data.
BlockReader	createBlockReader(long sessionId, long blockId, long offset) Creates a reader of an existing block to read data from this block.
default BlockReader	createBlockReader(long sessionId, long blockId, Protocol.OpenUfsBlockOptions options) Creates a reader of an existing block to read data from this block.
BlockWriter	createBlockWriter(long sessionId, long blockId) Creates a writer to write data to a temp block.
BlockStoreMeta	getBlockStoreMeta() Gets the metadata of the entire store in a snapshot.
BlockStoreMeta	getBlockStoreMetaFull() Similar as getBlockStoreMeta() except that this includes more information about the block store (e.g.
Optional<TempBlockMeta>	getTempBlockMeta(long blockId) Gets the temp metadata of a specific block from local storage.
Optional<BlockMeta>	getVolatileBlockMeta(long blockId) Gets the metadata of a block given its block id or empty if block does not exist.
boolean	hasBlockMeta(long blockId) Checks if the storage has a given block.
boolean	hasTempBlockMeta(long blockId) Checks if the storage has a given temp block.
void	moveBlock(long sessionId, long blockId, AllocateOptions moveOptions) Moves an existing block to a new location.
Optional<BlockLock>	pinBlock(long sessionId, long blockId) Pins the block indicating subsequent access.
void	registerBlockStoreEventListener(BlockStoreEventListener listener) Registers a BlockStoreEventListener to this block store.
void	removeBlock(long sessionId, long blockId) Removes an existing block.
void	removeInaccessibleStorage() Remove Storage directories that are no longer accessible.
void	requestSpace(long sessionId, long blockId, long additionalBytes) Requests to increase the size of a temp block.
void	updatePinnedInodes(Set<Long> inodes) Update the pinned inodes.

Methods inherited from interface java.io.Closeable

close

Method Detail

pinBlock

Optional<BlockLock> pinBlock(long sessionId,
                             long blockId)

Pins the block indicating subsequent access.

Parameters:

sessionId - the id of the session to lock this block

blockId - the id of the block to lock

Returns:

a lock of block to conveniently unpin the block later, or empty if the block does not exist

createBlock

TempBlockMeta createBlock(long sessionId,
                          long blockId,
                          AllocateOptions options)

Creates the metadata of a new block and assigns a temporary path (e.g., a subdir of the final location named after session id) to store its data. The location can be a location with specific tier and dir, or BlockStoreLocation.anyTier(), or BlockStoreLocation.anyDirInTier(String).

Before commit, all the data written to this block will be stored in the temp path and the block is only "visible" to its writer client.

Parameters:

sessionId - the id of the session

blockId - the id of the block to create

options - allocation options

Returns:

metadata of the temp block created

getVolatileBlockMeta

Optional<BlockMeta> getVolatileBlockMeta(long blockId)

Gets the metadata of a block given its block id or empty if block does not exist. This method does not require a lock id so the block is possible to be moved or removed after it returns.

Parameters:

blockId - the block id

Returns:

metadata of the block

getTempBlockMeta

Optional<TempBlockMeta> getTempBlockMeta(long blockId)

Gets the temp metadata of a specific block from local storage.

Parameters:

blockId - the id of the block

Returns:

metadata of the block if the temp block exists

commitBlock

void commitBlock(long sessionId,
                 long blockId,
                 boolean pinOnCreate)

Commits a temporary block to the local store. After commit, the block will be available in this block store for all clients to access. Since a temp block is "private" to the writer, this method requires no previously acquired lock.

Parameters:

sessionId - the id of the session

blockId - the id of a temp block

pinOnCreate - whether to pin block on create

commitBlockLocked

BlockLock commitBlockLocked(long sessionId,
                            long blockId,
                            boolean pinOnCreate)

Similar to commitBlock(long, long, boolean). It returns the block locked, so the caller is required to explicitly unlock the block. //TODO(Beinan): make this method to be private

Parameters:

sessionId - the id of the session

blockId - the id of a temp block

pinOnCreate - whether to pin block on create

Returns:

the lock

abortBlock

void abortBlock(long sessionId,
                long blockId)

Aborts a temporary block. The metadata of this block will not be added, its data will be deleted and the space will be reclaimed. Since a temp block is "private" to the writer, this requires no previously acquired lock.

Parameters:

sessionId - the id of the session

blockId - the id of a temp block

requestSpace

void requestSpace(long sessionId,
                  long blockId,
                  long additionalBytes)

Requests to increase the size of a temp block. Since a temp block is "private" to the writer client, this operation requires no previously acquired lock.

Parameters:

sessionId - the id of the session to request space

blockId - the id of the temp block

additionalBytes - the amount of more space to request in bytes, never be less than 0

createBlockWriter

BlockWriter createBlockWriter(long sessionId,
                              long blockId)

Creates a writer to write data to a temp block. Since the temp block is "private" to the writer, this operation requires no previously acquired lock.

Parameters:

sessionId - the id of the session to get the writer

blockId - the id of the temp block

Returns:

a BlockWriter instance on this block

createBlockReader

BlockReader createBlockReader(long sessionId,
                              long blockId,
                              long offset)
                       throws IOException

Creates a reader of an existing block to read data from this block.

This operation requires the lock id returned by a previously acquired pinBlock(long, long).

Parameters:

sessionId - the id of the session to get the reader

blockId - the id of an existing block

offset - the offset within the block

Returns:

a BlockReader instance on this block

Throws:

BlockDoesNotExistRuntimeException - if lockId is not found

IOException

createBlockReader

default BlockReader createBlockReader(long sessionId,
                                      long blockId,
                                      Protocol.OpenUfsBlockOptions options)

Creates a reader of an existing block to read data from this block.

The block reader will fetch the data from UFS when the data is not cached by the worker

Parameters:

sessionId - the id of the session to get the reader

blockId - the id of an existing block

options - the options for UFS fall-back

Returns:

a BlockReader instance on this block

moveBlock

void moveBlock(long sessionId,
               long blockId,
               AllocateOptions moveOptions)
        throws IOException

Moves an existing block to a new location.

Parameters:

sessionId - the id of the session to move a block

blockId - the id of an existing block

moveOptions - the options for move block

Throws:

IOException

removeBlock

void removeBlock(long sessionId,
                 long blockId)
          throws IOException

Removes an existing block. If the block can not be found in this store.

Parameters:

sessionId - the id of the session to remove a block

blockId - the id of an existing block

Throws:

IOException

accessBlock

void accessBlock(long sessionId,
                 long blockId)

Notifies the block store that a block was accessed so the block store could update accordingly the registered listeners such as evictor and allocator on block access.

Parameters:

sessionId - the id of the session to access a block

blockId - the id of an accessed block

getBlockStoreMeta

BlockStoreMeta getBlockStoreMeta()

Gets the metadata of the entire store in a snapshot. There is no guarantee the state will be consistent with the snapshot after this method is called. This function should be cheap since it is called for every block.

Returns:

store metadata

getBlockStoreMetaFull

BlockStoreMeta getBlockStoreMetaFull()

Similar as getBlockStoreMeta() except that this includes more information about the block store (e.g. blockId list). This is an expensive operation.

Returns:

full store metadata

hasBlockMeta

boolean hasBlockMeta(long blockId)

Checks if the storage has a given block.

Parameters:

blockId - the block id

Returns:

true if the block is contained, false otherwise

hasTempBlockMeta

boolean hasTempBlockMeta(long blockId)

Checks if the storage has a given temp block.

Parameters:

blockId - the temp block id

Returns:

true if the block is contained, false otherwise

cleanupSession

void cleanupSession(long sessionId)

Cleans up the data associated with a specific session (typically a dead session). Clean up entails unlocking the block locks of this session, reclaiming space of temp blocks created by this session, and deleting the session temporary folder.

Specified by:

cleanupSession in interface SessionCleanable

Parameters:

sessionId - the session id

registerBlockStoreEventListener

void registerBlockStoreEventListener(BlockStoreEventListener listener)

Registers a BlockStoreEventListener to this block store.

Parameters:

listener - the listener to those events

updatePinnedInodes

void updatePinnedInodes(Set<Long> inodes)

Update the pinned inodes.

Parameters:

inodes - a set of inodes that are currently pinned

removeInaccessibleStorage

void removeInaccessibleStorage()

Remove Storage directories that are no longer accessible.