alluxio.worker.block

Interface BlockStore

All Superinterfaces:

AutoCloseable, Closeable, SessionCleanable

All Known Implementing Classes:

TieredBlockStore

public interface BlockStore
extends SessionCleanable, Closeable

A blob store interface to represent the local storage managing and serving all the blocks in the local storage.

Method Summary

Modifier and Type	Method and Description
void	abortBlock(long sessionId, long blockId) Aborts a temporary block.
void	accessBlock(long sessionId, long blockId) Notifies the block store that a block was accessed so the block store could update accordingly the registered listeners such as evictor and allocator on block access.
boolean	checkStorage() Checks storage health.
void	cleanupSession(long sessionId) Cleans up the data associated with a specific session (typically a dead session).
void	commitBlock(long sessionId, long blockId, boolean pinOnCreate) Commits a temporary block to the local store.
long	commitBlockLocked(long sessionId, long blockId, boolean pinOnCreate) Similar to commitBlock(long, long, boolean).
TempBlockMeta	createBlock(long sessionId, long blockId, AllocateOptions options) Creates the metadata of a new block and assigns a temporary path (e.g., a subdir of the final location named after session id) to store its data.
void	freeSpace(long sessionId, long minContigiousBytes, long minAvailableBytes, BlockStoreLocation location) Frees space to make a specific amount of bytes available in the location.
BlockMeta	getBlockMeta(long sessionId, long blockId, long lockId) Gets the metadata of a specific block from local storage.
BlockReader	getBlockReader(long sessionId, long blockId, long lockId) Creates a reader of an existing block to read data from this block.
BlockStoreMeta	getBlockStoreMeta() Gets the metadata of the entire store in a snapshot.
BlockStoreMeta	getBlockStoreMetaFull() Similar as getBlockStoreMeta() except that this includes more information about the block store (e.g.
BlockWriter	getBlockWriter(long sessionId, long blockId) Creates a writer to write data to a temp block.
TempBlockMeta	getTempBlockMeta(long sessionId, long blockId) Gets the temp metadata of a specific block from local storage.
BlockMeta	getVolatileBlockMeta(long blockId) Gets the metadata of a block given its block id or throws BlockDoesNotExistException.
boolean	hasBlockMeta(long blockId) Checks if the storage has a given block.
long	lockBlock(long sessionId, long blockId) Locks an existing block and guards subsequent reads on this block.
long	lockBlockNoException(long sessionId, long blockId) Locks an existing block and guards subsequent reads on this block.
void	moveBlock(long sessionId, long blockId, AllocateOptions moveOptions) Moves an existing block to a new location.
void	moveBlock(long sessionId, long blockId, BlockStoreLocation oldLocation, AllocateOptions moveOptions) Moves an existing block to a new location.
void	registerBlockStoreEventListener(BlockStoreEventListener listener) Registers a BlockStoreEventListener to this block store.
void	removeBlock(long sessionId, long blockId) Removes an existing block.
void	removeBlock(long sessionId, long blockId, BlockStoreLocation location) Removes an existing block.
void	requestSpace(long sessionId, long blockId, long additionalBytes) Requests to increase the size of a temp block.
void	unlockBlock(long lockId) Releases an acquired block lock based on a lockId (returned by lockBlock(long, long).
boolean	unlockBlock(long sessionId, long blockId) Releases an acquired block lock based on a session id and block id.
void	updatePinnedInodes(Set<Long> inodes) Update the pinned inodes.

Methods inherited from interface java.io.Closeable

close

Method Detail

lockBlock

long lockBlock(long sessionId,
               long blockId)
        throws BlockDoesNotExistException

Locks an existing block and guards subsequent reads on this block.

Parameters:

sessionId - the id of the session to lock this block

blockId - the id of the block to lock

Returns:

the lock id (non-negative) if the lock is acquired successfully

Throws:

BlockDoesNotExistException - if block id can not be found, for example, evicted already

lockBlockNoException

long lockBlockNoException(long sessionId,
                          long blockId)

Locks an existing block and guards subsequent reads on this block. If the lock fails, return BlockLockManager.INVALID_LOCK_ID.

Parameters:

sessionId - the id of the session to lock this block

blockId - the id of the block to lock

Returns:

the lock id (non-negative) that uniquely identifies the lock obtained or BlockLockManager.INVALID_LOCK_ID if it failed to lock

unlockBlock

void unlockBlock(long lockId)
          throws BlockDoesNotExistException

Releases an acquired block lock based on a lockId (returned by lockBlock(long, long).

Parameters:

lockId - the id of the lock returned by lockBlock(long, long)

Throws:

BlockDoesNotExistException - if lockId can not be found

unlockBlock

boolean unlockBlock(long sessionId,
                    long blockId)

Releases an acquired block lock based on a session id and block id. TODO(calvin): temporary, will be removed after changing client side code.

Parameters:

sessionId - the id of the session to lock this block

blockId - the id of the block to lock

Returns:

false if it fails to unlock due to the lock is not found

createBlock

TempBlockMeta createBlock(long sessionId,
                          long blockId,
                          AllocateOptions options)
                   throws BlockAlreadyExistsException,
                          WorkerOutOfSpaceException,
                          IOException

Creates the metadata of a new block and assigns a temporary path (e.g., a subdir of the final location named after session id) to store its data. The location can be a location with specific tier and dir, or BlockStoreLocation.anyTier(), or BlockStoreLocation.anyDirInTier(String).

Before commit, all the data written to this block will be stored in the temp path and the block is only "visible" to its writer client.

Parameters:

sessionId - the id of the session

blockId - the id of the block to create

options - allocation options

Returns:

metadata of the temp block created

Throws:

IllegalArgumentException - if location does not belong to tiered storage

BlockAlreadyExistsException - if block id already exists, either temporary or committed, or block in eviction plan already exists

WorkerOutOfSpaceException - if this Store has no more space than the initialBlockSize

IOException

getVolatileBlockMeta

BlockMeta getVolatileBlockMeta(long blockId)
                        throws BlockDoesNotExistException

Gets the metadata of a block given its block id or throws BlockDoesNotExistException. This method does not require a lock id so the block is possible to be moved or removed after it returns.

Parameters:

blockId - the block id

Returns:

metadata of the block

Throws:

BlockDoesNotExistException - if no BlockMeta for this block id is found

getBlockMeta

BlockMeta getBlockMeta(long sessionId,
                       long blockId,
                       long lockId)
                throws BlockDoesNotExistException,
                       InvalidWorkerStateException

Gets the metadata of a specific block from local storage.

This method requires the lock id returned by a previously acquired lockBlock(long, long).

Parameters:

sessionId - the id of the session to get this file

blockId - the id of the block

lockId - the id of the lock

Returns:

metadata of the block

Throws:

BlockDoesNotExistException - if the block id can not be found in committed blocks or lockId can not be found

InvalidWorkerStateException - if session id or block id is not the same as that in the LockRecord of lockId

getTempBlockMeta

@Nullable
TempBlockMeta getTempBlockMeta(long sessionId,
                                         long blockId)

Gets the temp metadata of a specific block from local storage.

Parameters:

sessionId - the id of the session to get this file

blockId - the id of the block

Returns:

metadata of the block or null if the temp block does not exist

commitBlock

void commitBlock(long sessionId,
                 long blockId,
                 boolean pinOnCreate)
          throws BlockAlreadyExistsException,
                 BlockDoesNotExistException,
                 InvalidWorkerStateException,
                 IOException,
                 WorkerOutOfSpaceException

Commits a temporary block to the local store. After commit, the block will be available in this block store for all clients to access. Since a temp block is "private" to the writer, this method requires no previously acquired lock.

Parameters:

sessionId - the id of the session

blockId - the id of a temp block

pinOnCreate - whether to pin block on create

Throws:

BlockAlreadyExistsException - if block id already exists in committed blocks

BlockDoesNotExistException - if the temporary block can not be found

InvalidWorkerStateException - if block id does not belong to session id

WorkerOutOfSpaceException - if there is no more space left to hold the block

IOException

commitBlockLocked

long commitBlockLocked(long sessionId,
                       long blockId,
                       boolean pinOnCreate)
                throws BlockAlreadyExistsException,
                       BlockDoesNotExistException,
                       InvalidWorkerStateException,
                       IOException,
                       WorkerOutOfSpaceException

Similar to commitBlock(long, long, boolean). It returns the block locked, so the caller is required to explicitly unlock the block.

Parameters:

sessionId - the id of the session

blockId - the id of a temp block

pinOnCreate - whether to pin block on create

Returns:

the lock id

Throws:

BlockAlreadyExistsException - if block id already exists in committed blocks

BlockDoesNotExistException - if the temporary block can not be found

InvalidWorkerStateException - if block id does not belong to session id

WorkerOutOfSpaceException - if there is no more space left to hold the block

IOException

abortBlock

void abortBlock(long sessionId,
                long blockId)
         throws BlockAlreadyExistsException,
                BlockDoesNotExistException,
                InvalidWorkerStateException,
                IOException

Aborts a temporary block. The metadata of this block will not be added, its data will be deleted and the space will be reclaimed. Since a temp block is "private" to the writer, this requires no previously acquired lock.

Parameters:

sessionId - the id of the session

blockId - the id of a temp block

Throws:

BlockAlreadyExistsException - if block id already exists in committed blocks

BlockDoesNotExistException - if the temporary block can not be found

InvalidWorkerStateException - if block id does not belong to session id

IOException

requestSpace

void requestSpace(long sessionId,
                  long blockId,
                  long additionalBytes)
           throws BlockDoesNotExistException,
                  WorkerOutOfSpaceException,
                  IOException

Requests to increase the size of a temp block. Since a temp block is "private" to the writer client, this operation requires no previously acquired lock.

Parameters:

sessionId - the id of the session to request space

blockId - the id of the temp block

additionalBytes - the amount of more space to request in bytes, never be less than 0

Throws:

BlockDoesNotExistException - if block id can not be found, or some block in eviction plan cannot be found

WorkerOutOfSpaceException - if requested space can not be satisfied

IOException

getBlockWriter

BlockWriter getBlockWriter(long sessionId,
                           long blockId)
                    throws BlockDoesNotExistException,
                           BlockAlreadyExistsException,
                           InvalidWorkerStateException,
                           IOException

Creates a writer to write data to a temp block. Since the temp block is "private" to the writer, this operation requires no previously acquired lock.

Parameters:

sessionId - the id of the session to get the writer

blockId - the id of the temp block

Returns:

a BlockWriter instance on this block

Throws:

BlockDoesNotExistException - if the block can not be found

BlockAlreadyExistsException - if a committed block with the same ID exists

InvalidWorkerStateException - if the worker state is invalid

IOException

getBlockReader

BlockReader getBlockReader(long sessionId,
                           long blockId,
                           long lockId)
                    throws BlockDoesNotExistException,
                           InvalidWorkerStateException,
                           IOException

Creates a reader of an existing block to read data from this block.

This operation requires the lock id returned by a previously acquired lockBlock(long, long).

Parameters:

sessionId - the id of the session to get the reader

blockId - the id of an existing block

lockId - the id of the lock returned by lockBlock(long, long)

Returns:

a BlockReader instance on this block

Throws:

BlockDoesNotExistException - if lockId is not found

InvalidWorkerStateException - if session id or block id is not the same as that in the LockRecord of lockId

IOException

moveBlock

void moveBlock(long sessionId,
               long blockId,
               AllocateOptions moveOptions)
        throws BlockDoesNotExistException,
               BlockAlreadyExistsException,
               InvalidWorkerStateException,
               WorkerOutOfSpaceException,
               IOException

Moves an existing block to a new location.

Parameters:

sessionId - the id of the session to move a block

blockId - the id of an existing block

moveOptions - the options for move

Throws:

IllegalArgumentException - if newLocation does not belong to the tiered storage

BlockDoesNotExistException - if block id can not be found

BlockAlreadyExistsException - if block id already exists in committed blocks of the newLocation

InvalidWorkerStateException - if block id has not been committed

WorkerOutOfSpaceException - if newLocation does not have enough extra space to hold the block

IOException

moveBlock

void moveBlock(long sessionId,
               long blockId,
               BlockStoreLocation oldLocation,
               AllocateOptions moveOptions)
        throws BlockDoesNotExistException,
               BlockAlreadyExistsException,
               InvalidWorkerStateException,
               WorkerOutOfSpaceException,
               IOException

Moves an existing block to a new location.

Parameters:

sessionId - the id of the session to remove a block

blockId - the id of an existing block

oldLocation - the location of the source

moveOptions - the options for move

Throws:

IllegalArgumentException - if newLocation does not belong to the tiered storage

BlockDoesNotExistException - if block id can not be found

BlockAlreadyExistsException - if block id already exists in committed blocks of the newLocation

InvalidWorkerStateException - if block id has not been committed

WorkerOutOfSpaceException - if newLocation does not have enough extra space to hold the block

IOException

removeBlock

void removeBlock(long sessionId,
                 long blockId)
          throws InvalidWorkerStateException,
                 BlockDoesNotExistException,
                 IOException

Removes an existing block. If the block can not be found in this store.

Parameters:

sessionId - the id of the session to remove a block

blockId - the id of an existing block

Throws:

InvalidWorkerStateException - if block id has not been committed

BlockDoesNotExistException - if block can not be found

IOException

removeBlock

void removeBlock(long sessionId,
                 long blockId,
                 BlockStoreLocation location)
          throws InvalidWorkerStateException,
                 BlockDoesNotExistException,
                 IOException

Removes an existing block. If the block can not be found in this store.

Parameters:

sessionId - the id of the session to move a block

blockId - the id of an existing block

location - the location of the block

Throws:

InvalidWorkerStateException - if block id has not been committed

BlockDoesNotExistException - if block can not be found

IOException

accessBlock

void accessBlock(long sessionId,
                 long blockId)
          throws BlockDoesNotExistException

Notifies the block store that a block was accessed so the block store could update accordingly the registered listeners such as evictor and allocator on block access.

Parameters:

sessionId - the id of the session to access a block

blockId - the id of an accessed block

Throws:

BlockDoesNotExistException - if the block id is not found

getBlockStoreMeta

BlockStoreMeta getBlockStoreMeta()

Gets the metadata of the entire store in a snapshot. There is no guarantee the state will be consistent with the snapshot after this method is called. This function should be cheap since it is called for every block.

Returns:

store metadata

getBlockStoreMetaFull

BlockStoreMeta getBlockStoreMetaFull()

Similar as getBlockStoreMeta() except that this includes more information about the block store (e.g. blockId list). This is an expensive operation.

Returns:

full store metadata

hasBlockMeta

boolean hasBlockMeta(long blockId)

Checks if the storage has a given block.

Parameters:

blockId - the block id

Returns:

true if the block is contained, false otherwise

cleanupSession

void cleanupSession(long sessionId)

Cleans up the data associated with a specific session (typically a dead session). Clean up entails unlocking the block locks of this session, reclaiming space of temp blocks created by this session, and deleting the session temporary folder.

Specified by:

cleanupSession in interface SessionCleanable

Parameters:

sessionId - the session id

freeSpace

void freeSpace(long sessionId,
               long minContigiousBytes,
               long minAvailableBytes,
               BlockStoreLocation location)
        throws WorkerOutOfSpaceException,
               BlockDoesNotExistException,
               IOException

Frees space to make a specific amount of bytes available in the location.

Parameters:

sessionId - the session id

minContigiousBytes - the minimum amount of contigious free space in bytes

minAvailableBytes - the maximum amount of free space in bytes

location - the location to free space

Throws:

WorkerOutOfSpaceException - if there is not enough space to fulfill minimum requirement

BlockDoesNotExistException - if blocks in EvictionPlan can not be found

IOException

registerBlockStoreEventListener

void registerBlockStoreEventListener(BlockStoreEventListener listener)

Registers a BlockStoreEventListener to this block store.

Parameters:

listener - the listener to those events

updatePinnedInodes

void updatePinnedInodes(Set<Long> inodes)

Update the pinned inodes.

Parameters:

inodes - a set of inodes that are currently pinned

checkStorage

boolean checkStorage()

Checks storage health.

Returns:

true if at least one storage path failed check and is removed, false otherwise