alluxio.worker.block

Interface BlockStore

All Superinterfaces:

AutoCloseable, Closeable, SessionCleanable

public interface BlockStore
extends Closeable, SessionCleanable

An abstraction of block store on worker.

Method Summary

Modifier and Type	Method and Description
void	abortBlock(long sessionId, long blockId) Aborts a temporary block.
void	accessBlock(long sessionId, long blockId) Notifies the block store that a block was accessed so the block store could update accordingly the registered listeners such as evictor and allocator on block access.
void	commitBlock(long sessionId, long blockId, boolean pinOnCreate) Commits a block to Alluxio managed space.
String	createBlock(long sessionId, long blockId, int tier, CreateBlockOptions createBlockOptions) Creates a block in Alluxio managed space.
BlockReader	createBlockReader(long sessionId, long blockId, long lockId) Creates a reader of an existing block to read data from this block.
BlockReader	createBlockReader(long sessionId, long blockId, long offset, boolean positionShort, Protocol.OpenUfsBlockOptions options) Creates the block reader to read from Alluxio block or UFS block.
BlockWriter	createBlockWriter(long sessionId, long blockId) Creates a BlockWriter for an existing temporary block which is already created by createBlock(long, long, int, alluxio.worker.block.CreateBlockOptions).
BlockReader	createUfsBlockReader(long sessionId, long blockId, long offset, boolean positionShort, Protocol.OpenUfsBlockOptions options) Creates a block reader to read a UFS block starting from given block offset.
BlockMeta	getBlockMeta(long sessionId, long blockId, long lockId) Gets the metadata of a specific block from local storage.
BlockStoreMeta	getBlockStoreMeta() Gets the metadata of the entire store in a snapshot.
BlockStoreMeta	getBlockStoreMetaFull() Similar as getBlockStoreMeta() except that this includes more information about the block store (e.g.
BlockWriter	getBlockWriter(long sessionId, long blockId) Creates a writer to write data to a temp block.
Optional<TempBlockMeta>	getTempBlockMeta(long blockId) Gets the temp metadata of a specific block from local storage.
Optional<BlockMeta>	getVolatileBlockMeta(long blockId) Gets the metadata of a block given its block id or empty if block does not exist.
boolean	hasBlockMeta(long blockId) Checks if the storage has a given block.
boolean	hasTempBlockMeta(long blockId) Checks if the storage has a given temp block.
default void	initialize() Initialize the block store.
CompletableFuture<List<BlockStatus>>	load(List<Block> fileBlocks, UfsReadOptions options) Load blocks into alluxio.
long	lockBlock(long sessionId, long blockId) Locks an existing block and guards subsequent reads on this block.
long	lockBlockNoException(long sessionId, long blockId) Locks an existing block and guards subsequent reads on this block.
void	moveBlock(long sessionId, long blockId, AllocateOptions moveOptions) Moves an existing block to a new location.
Optional<BlockLock>	pinBlock(long sessionId, long blockId) Pins the block indicating subsequent access.
void	registerBlockStoreEventListener(BlockStoreEventListener listener) Registers a BlockStoreEventListener to this block store.
void	removeBlock(long sessionId, long blockId) Removes an existing block.
void	removeInaccessibleStorage() Remove Storage directories that are no longer accessible.
void	requestSpace(long sessionId, long blockId, long additionalBytes) Requests to increase the size of a temp block.
void	unlockBlock(long lockId) Releases an acquired block lock based on a lockId (returned by lockBlock(long, long).
boolean	unlockBlock(long sessionId, long blockId) Releases an acquired block lock based on a session id and block id.
void	unpinBlock(BlockLock lock) Unpins an accessed block based on the id (returned by pinBlock(long, long)).
void	updatePinnedInodes(Set<Long> inodes) Update the pinned inodes.

Methods inherited from interface java.io.Closeable

close

Methods inherited from interface alluxio.worker.SessionCleanable

cleanupSession

Method Detail

initialize

default void initialize()

Initialize the block store.

abortBlock

void abortBlock(long sessionId,
                long blockId)

Aborts a temporary block. The metadata of this block will not be added, its data will be deleted and the space will be reclaimed. Since a temp block is "private" to the writer, this requires no previously acquired lock.

Parameters:

sessionId - the id of the session

blockId - the id of a temp block

accessBlock

void accessBlock(long sessionId,
                 long blockId)

Notifies the block store that a block was accessed so the block store could update accordingly the registered listeners such as evictor and allocator on block access. //TODO(beinan): looks like we should not expose this method except the test

Parameters:

sessionId - the id of the session to access a block

blockId - the id of an accessed block

commitBlock

void commitBlock(long sessionId,
                 long blockId,
                 boolean pinOnCreate)

Commits a block to Alluxio managed space. The block must be temporary. The block will not be persisted or accessible before commitBlock succeeds.

Parameters:

sessionId - the id of the client

blockId - the id of the block to commit

pinOnCreate - whether to pin block on create

createBlock

String createBlock(long sessionId,
                   long blockId,
                   int tier,
                   CreateBlockOptions createBlockOptions)

Creates a block in Alluxio managed space. Calls createBlockWriter(long, long) to get a writer for writing to the block. The block will be temporary until it is committed by commitBlock(long, long, boolean) . Throws an IllegalArgumentException if the location does not belong to tiered storage.

Parameters:

sessionId - the id of the client

blockId - the id of the block to create

tier - the tier to place the new block in BlockStoreLocation.ANY_TIER for any tier

createBlockOptions - the createBlockOptions

Returns:

a string representing the path to the local file

createBlockReader

BlockReader createBlockReader(long sessionId,
                              long blockId,
                              long offset,
                              boolean positionShort,
                              Protocol.OpenUfsBlockOptions options)
                       throws IOException

Creates the block reader to read from Alluxio block or UFS block. Owner of this block reader must close it or lock will leak.

Parameters:

sessionId - the client session ID

blockId - the ID of the UFS block to read

offset - the offset within the block

positionShort - whether the operation is using positioned read to a small buffer size

options - the options

Returns:

a block reader to read data from

Throws:

IOException - if it fails to get block reader

createBlockReader

BlockReader createBlockReader(long sessionId,
                              long blockId,
                              long lockId)
                       throws BlockDoesNotExistException,
                              IOException

Creates a reader of an existing block to read data from this block.

This operation requires the lock id returned by a previously acquired lockBlock(long, long).

Parameters:

sessionId - the id of the session to get the reader

blockId - the id of an existing block

lockId - the id of the lock returned by lockBlock(long, long)

Returns:

a BlockReader instance on this block

Throws:

BlockDoesNotExistException - if lockId is not found

IOException

createUfsBlockReader

BlockReader createUfsBlockReader(long sessionId,
                                 long blockId,
                                 long offset,
                                 boolean positionShort,
                                 Protocol.OpenUfsBlockOptions options)
                          throws IOException

Creates a block reader to read a UFS block starting from given block offset. Owner of this block reader must close it to clean up state.

Parameters:

sessionId - the client session ID

blockId - the ID of the UFS block to read

offset - the offset within the block

positionShort - whether the operation is using positioned read to a small buffer size

options - the options

Returns:

the block reader instance

Throws:

IOException - if it fails to get block reader

createBlockWriter

BlockWriter createBlockWriter(long sessionId,
                              long blockId)
                       throws IOException

Creates a BlockWriter for an existing temporary block which is already created by createBlock(long, long, int, alluxio.worker.block.CreateBlockOptions).

Parameters:

sessionId - the id of the client

blockId - the id of the block to be opened for writing

Returns:

the block writer for the local block file

Throws:

IOException

getBlockStoreMeta

BlockStoreMeta getBlockStoreMeta()

Gets the metadata of the entire store in a snapshot. There is no guarantee the state will be consistent with the snapshot after this method is called. This function should be cheap since it is called for every block.

Returns:

store metadata

getBlockStoreMetaFull

BlockStoreMeta getBlockStoreMetaFull()

Similar as getBlockStoreMeta() except that this includes more information about the block store (e.g. blockId list). This is an expensive operation.

Returns:

full store metadata

getTempBlockMeta

Optional<TempBlockMeta> getTempBlockMeta(long blockId)

Gets the temp metadata of a specific block from local storage.

Parameters:

blockId - the id of the block

Returns:

metadata of the block if the temp block exists

hasBlockMeta

boolean hasBlockMeta(long blockId)

Checks if the storage has a given block.

Parameters:

blockId - the block id

Returns:

true if the block is contained, false otherwise

hasTempBlockMeta

boolean hasTempBlockMeta(long blockId)

Checks if the storage has a given temp block.

Parameters:

blockId - the temp block id

Returns:

true if the block is contained, false otherwise

getVolatileBlockMeta

Optional<BlockMeta> getVolatileBlockMeta(long blockId)

Gets the metadata of a block given its block id or empty if block does not exist. This method does not require a lock id so the block is possible to be moved or removed after it returns.

Parameters:

blockId - the block id

Returns:

metadata of the block

moveBlock

void moveBlock(long sessionId,
               long blockId,
               AllocateOptions moveOptions)
        throws IOException

Moves an existing block to a new location.

Parameters:

sessionId - the id of the session to move a block

blockId - the id of an existing block

moveOptions - the options for move

Throws:

IOException

pinBlock

Optional<BlockLock> pinBlock(long sessionId,
                             long blockId)

Pins the block indicating subsequent access.

Parameters:

sessionId - the id of the session to lock this block

blockId - the id of the block to lock

Returns:

a lock of block to conveniently unpin the block later, or empty if the block does not exist

unpinBlock

void unpinBlock(BlockLock lock)

Unpins an accessed block based on the id (returned by pinBlock(long, long)).

Parameters:

lock - the lock returned by pinBlock(long, long)

updatePinnedInodes

void updatePinnedInodes(Set<Long> inodes)

Update the pinned inodes.

Parameters:

inodes - a set of inodes that are currently pinned

registerBlockStoreEventListener

void registerBlockStoreEventListener(BlockStoreEventListener listener)

Registers a BlockStoreEventListener to this block store.

Parameters:

listener - the listener to those events

removeBlock

void removeBlock(long sessionId,
                 long blockId)
          throws IOException

Removes an existing block. If the block can not be found in this store.

Parameters:

sessionId - the id of the session to remove a block

blockId - the id of an existing block

Throws:

IOException

removeInaccessibleStorage

void removeInaccessibleStorage()

Remove Storage directories that are no longer accessible.

requestSpace

void requestSpace(long sessionId,
                  long blockId,
                  long additionalBytes)

Requests to increase the size of a temp block. Since a temp block is "private" to the writer client, this operation requires no previously acquired lock.

Parameters:

sessionId - the id of the session to request space

blockId - the id of the temp block

additionalBytes - the amount of more space to request in bytes, never be less than 0

load

CompletableFuture<List<BlockStatus>> load(List<Block> fileBlocks,
                                          UfsReadOptions options)

Load blocks into alluxio.

Parameters:

fileBlocks - list of fileBlocks, one file blocks contains blocks belong to one file

options - read ufs options

Returns:

future of load status for failed blocks

getBlockMeta

BlockMeta getBlockMeta(long sessionId,
                       long blockId,
                       long lockId)
                throws BlockDoesNotExistException,
                       InvalidWorkerStateException

Gets the metadata of a specific block from local storage.

This method requires the lock id returned by a previously acquired lockBlock(long, long).

Parameters:

sessionId - the id of the session to get this file

blockId - the id of the block

lockId - the id of the lock

Returns:

metadata of the block

Throws:

BlockDoesNotExistException - if the block id can not be found in committed blocks or lockId can not be found

InvalidWorkerStateException - if session id or block id is not the same as that in the LockRecord of lockId

lockBlock

long lockBlock(long sessionId,
               long blockId)
        throws BlockDoesNotExistException

Locks an existing block and guards subsequent reads on this block.

Parameters:

sessionId - the id of the session to lock this block

blockId - the id of the block to lock

Returns:

the lock id (non-negative) if the lock is acquired successfully

Throws:

BlockDoesNotExistException - if block id can not be found, for example, evicted already

lockBlockNoException

long lockBlockNoException(long sessionId,
                          long blockId)

Locks an existing block and guards subsequent reads on this block. If the lock fails, return BlockLockManager#INVALID_LOCK_ID.

Parameters:

sessionId - the id of the session to lock this block

blockId - the id of the block to lock

Returns:

the lock id (non-negative) that uniquely identifies the lock obtained or BlockLockManager#INVALID_LOCK_ID if it failed to lock

unlockBlock

void unlockBlock(long lockId)
          throws BlockDoesNotExistException

Releases an acquired block lock based on a lockId (returned by lockBlock(long, long).

Parameters:

lockId - the id of the lock returned by lockBlock(long, long)

Throws:

BlockDoesNotExistException - if lockId can not be found

unlockBlock

boolean unlockBlock(long sessionId,
                    long blockId)

Releases an acquired block lock based on a session id and block id. TODO(calvin): temporary, will be removed after changing client side code.

Parameters:

sessionId - the id of the session to lock this block

blockId - the id of the block to lock

Returns:

false if it fails to unlock due to the lock is not found

getBlockWriter

BlockWriter getBlockWriter(long sessionId,
                           long blockId)
                    throws BlockDoesNotExistException,
                           BlockAlreadyExistsException,
                           InvalidWorkerStateException,
                           IOException

Creates a writer to write data to a temp block. Since the temp block is "private" to the writer, this operation requires no previously acquired lock.

Parameters:

sessionId - the id of the session to get the writer

blockId - the id of the temp block

Returns:

a BlockWriter instance on this block

Throws:

BlockDoesNotExistException - if the block can not be found

BlockAlreadyExistsException - if a committed block with the same ID exists

InvalidWorkerStateException - if the worker state is invalid

IOException