@NotThreadSafe public class TieredBlockStore extends Object implements BlockStore
This class is thread-safe, using the following lock hierarchy to ensure thread-safety:
mLockManager
. This block lock is a
read/write lock, guarding both the metadata operations and the following I/O on this block. It
coordinates different threads (clients) when accessing the same block concurrently.mMetaManager
and guarded by mMetadataLock
. This is also a read/write lock and
coordinates different threads (clients) when accessing the shared data structure for metadata.
createBlock(long, long, alluxio.worker.block.BlockStoreLocation, long)
does not acquire the block lock, because it only creates a
temp block which is only visible to its writer before committed (thus no concurrent access).abortBlock(long, long)
does not acquire the block lock, because only
temporary blocks can be aborted, and they are only visible to their writers (thus no concurrent
access).
freeSpaceInternal(long, long, alluxio.worker.block.BlockStoreLocation, alluxio.worker.block.evictor.Evictor.Mode)
and it is on the basis of best effort. For
operations that may trigger this eviction (e.g., move, create, requestSpace), retry is usedConstructor and Description |
---|
TieredBlockStore()
Creates a new instance of
TieredBlockStore . |
Modifier and Type | Method and Description |
---|---|
void |
abortBlock(long sessionId,
long blockId)
Aborts a temporary block.
|
void |
accessBlock(long sessionId,
long blockId)
Notifies the block store that a block was accessed so the block store could update accordingly
the registered listeners such as evictor and allocator on block access.
|
boolean |
checkStorage()
Checks storage health.
|
void |
cleanupSession(long sessionId)
Cleans up the data associated with a specific session (typically a dead session).
|
void |
commitBlock(long sessionId,
long blockId,
boolean pinOnCreate)
Commits a temporary block to the local store.
|
TempBlockMeta |
createBlock(long sessionId,
long blockId,
BlockStoreLocation location,
long initialBlockSize)
Creates the metadata of a new block and assigns a temporary path (e.g., a subdir of the final
location named after session id) to store its data.
|
void |
freeSpace(long sessionId,
long availableBytes,
BlockStoreLocation location)
Frees space to make a specific amount of bytes available in a best-effort way in the location.
|
BlockMeta |
getBlockMeta(long sessionId,
long blockId,
long lockId)
Gets the metadata of a specific block from local storage.
|
BlockReader |
getBlockReader(long sessionId,
long blockId,
long lockId)
Creates a reader of an existing block to read data from this block.
|
BlockStoreMeta |
getBlockStoreMeta()
Gets the metadata of the entire store in a snapshot.
|
BlockStoreMeta |
getBlockStoreMetaFull()
Similar as
BlockStore.getBlockStoreMeta() except that this includes
more information about the block store (e.g. |
BlockWriter |
getBlockWriter(long sessionId,
long blockId)
Creates a writer to write data to a temp block.
|
TempBlockMeta |
getTempBlockMeta(long sessionId,
long blockId)
Gets the temp metadata of a specific block from local storage.
|
BlockMeta |
getVolatileBlockMeta(long blockId)
Gets the metadata of a block given its block id or throws
BlockDoesNotExistException . |
boolean |
hasBlockMeta(long blockId)
Checks if the storage has a given block.
|
long |
lockBlock(long sessionId,
long blockId)
Locks an existing block and guards subsequent reads on this block.
|
long |
lockBlockNoException(long sessionId,
long blockId)
Locks an existing block and guards subsequent reads on this block.
|
void |
moveBlock(long sessionId,
long blockId,
BlockStoreLocation newLocation)
Moves an existing block to a new location.
|
void |
moveBlock(long sessionId,
long blockId,
BlockStoreLocation oldLocation,
BlockStoreLocation newLocation)
Moves an existing block to a new location.
|
void |
registerBlockStoreEventListener(BlockStoreEventListener listener)
Registers a
BlockStoreEventListener to this block store. |
void |
removeBlock(long sessionId,
long blockId)
Removes an existing block.
|
void |
removeBlock(long sessionId,
long blockId,
BlockStoreLocation location)
Removes an existing block.
|
void |
removeDir(StorageDir dir)
Removes a storage directory.
|
void |
requestSpace(long sessionId,
long blockId,
long additionalBytes)
Requests to increase the size of a temp block.
|
void |
unlockBlock(long lockId)
Releases an acquired block lock based on a lockId (returned by
BlockStore.lockBlock(long, long) . |
boolean |
unlockBlock(long sessionId,
long blockId)
Releases an acquired block lock based on a session id and block id.
|
void |
updatePinnedInodes(Set<Long> inodes)
Updates the pinned blocks.
|
public TieredBlockStore()
TieredBlockStore
.public long lockBlock(long sessionId, long blockId) throws BlockDoesNotExistException
BlockStore
lockBlock
in interface BlockStore
sessionId
- the id of the session to lock this blockblockId
- the id of the block to lockBlockDoesNotExistException
- if block id can not be found, for example, evicted alreadypublic long lockBlockNoException(long sessionId, long blockId)
BlockStore
BlockLockManager.INVALID_LOCK_ID
.lockBlockNoException
in interface BlockStore
sessionId
- the id of the session to lock this blockblockId
- the id of the block to lockBlockLockManager.INVALID_LOCK_ID
if it failed to lockpublic void unlockBlock(long lockId) throws BlockDoesNotExistException
BlockStore
BlockStore.lockBlock(long, long)
.unlockBlock
in interface BlockStore
lockId
- the id of the lock returned by BlockStore.lockBlock(long, long)
BlockDoesNotExistException
- if lockId can not be foundpublic boolean unlockBlock(long sessionId, long blockId)
BlockStore
unlockBlock
in interface BlockStore
sessionId
- the id of the session to lock this blockblockId
- the id of the block to lockpublic BlockWriter getBlockWriter(long sessionId, long blockId) throws BlockDoesNotExistException, BlockAlreadyExistsException, InvalidWorkerStateException, IOException
BlockStore
getBlockWriter
in interface BlockStore
sessionId
- the id of the session to get the writerblockId
- the id of the temp blockBlockWriter
instance on this blockBlockDoesNotExistException
- if the block can not be foundBlockAlreadyExistsException
- if a committed block with the same ID existsInvalidWorkerStateException
- if the worker state is invalidIOException
public BlockReader getBlockReader(long sessionId, long blockId, long lockId) throws BlockDoesNotExistException, InvalidWorkerStateException, IOException
BlockStore
This operation requires the lock id returned by a previously acquired
BlockStore.lockBlock(long, long)
.
getBlockReader
in interface BlockStore
sessionId
- the id of the session to get the readerblockId
- the id of an existing blocklockId
- the id of the lock returned by BlockStore.lockBlock(long, long)
BlockReader
instance on this blockBlockDoesNotExistException
- if lockId is not foundInvalidWorkerStateException
- if session id or block id is not the same as that in the
LockRecord of lockIdIOException
public TempBlockMeta createBlock(long sessionId, long blockId, BlockStoreLocation location, long initialBlockSize) throws BlockAlreadyExistsException, WorkerOutOfSpaceException, IOException
BlockStore
BlockStoreLocation.anyTier()
, or
BlockStoreLocation.anyDirInTier(String)
.
Before commit, all the data written to this block will be stored in the temp path and the block is only "visible" to its writer client.
createBlock
in interface BlockStore
sessionId
- the id of the sessionblockId
- the id of the block to createlocation
- location to create this blockinitialBlockSize
- initial size of this block in bytesBlockAlreadyExistsException
- if block id already exists, either temporary or committed,
or block in eviction plan already existsWorkerOutOfSpaceException
- if this Store has no more space than the initialBlockSizeIOException
public BlockMeta getVolatileBlockMeta(long blockId) throws BlockDoesNotExistException
BlockStore
BlockDoesNotExistException
.
This method does not require a lock id so the block is possible to be moved or removed after it
returns.getVolatileBlockMeta
in interface BlockStore
blockId
- the block idBlockDoesNotExistException
- if no BlockMeta for this block id is foundpublic BlockMeta getBlockMeta(long sessionId, long blockId, long lockId) throws BlockDoesNotExistException, InvalidWorkerStateException
BlockStore
This method requires the lock id returned by a previously acquired
BlockStore.lockBlock(long, long)
.
getBlockMeta
in interface BlockStore
sessionId
- the id of the session to get this fileblockId
- the id of the blocklockId
- the id of the lockBlockDoesNotExistException
- if the block id can not be found in committed blocks or
lockId can not be foundInvalidWorkerStateException
- if session id or block id is not the same as that in the
LockRecord of lockIdpublic TempBlockMeta getTempBlockMeta(long sessionId, long blockId)
BlockStore
getTempBlockMeta
in interface BlockStore
sessionId
- the id of the session to get this fileblockId
- the id of the blockpublic void commitBlock(long sessionId, long blockId, boolean pinOnCreate) throws BlockAlreadyExistsException, InvalidWorkerStateException, BlockDoesNotExistException, IOException
BlockStore
commitBlock
in interface BlockStore
sessionId
- the id of the sessionblockId
- the id of a temp blockpinOnCreate
- whether to pin block on createBlockAlreadyExistsException
- if block id already exists in committed blocksInvalidWorkerStateException
- if block id does not belong to session idBlockDoesNotExistException
- if the temporary block can not be foundIOException
public void abortBlock(long sessionId, long blockId) throws BlockAlreadyExistsException, BlockDoesNotExistException, InvalidWorkerStateException, IOException
BlockStore
abortBlock
in interface BlockStore
sessionId
- the id of the sessionblockId
- the id of a temp blockBlockAlreadyExistsException
- if block id already exists in committed blocksBlockDoesNotExistException
- if the temporary block can not be foundInvalidWorkerStateException
- if block id does not belong to session idIOException
public void requestSpace(long sessionId, long blockId, long additionalBytes) throws BlockDoesNotExistException, WorkerOutOfSpaceException, IOException
BlockStore
requestSpace
in interface BlockStore
sessionId
- the id of the session to request spaceblockId
- the id of the temp blockadditionalBytes
- the amount of more space to request in bytes, never be less than 0BlockDoesNotExistException
- if block id can not be found, or some block in eviction plan
cannot be foundWorkerOutOfSpaceException
- if requested space can not be satisfiedIOException
public void moveBlock(long sessionId, long blockId, BlockStoreLocation newLocation) throws BlockDoesNotExistException, BlockAlreadyExistsException, InvalidWorkerStateException, WorkerOutOfSpaceException, IOException
BlockStore
moveBlock
in interface BlockStore
sessionId
- the id of the session to move a blockblockId
- the id of an existing blocknewLocation
- the location of the destinationBlockDoesNotExistException
- if block id can not be foundBlockAlreadyExistsException
- if block id already exists in committed blocks of the
newLocationInvalidWorkerStateException
- if block id has not been committedWorkerOutOfSpaceException
- if newLocation does not have enough extra space to hold the
blockIOException
public void moveBlock(long sessionId, long blockId, BlockStoreLocation oldLocation, BlockStoreLocation newLocation) throws BlockDoesNotExistException, BlockAlreadyExistsException, InvalidWorkerStateException, WorkerOutOfSpaceException, IOException
BlockStore
moveBlock
in interface BlockStore
sessionId
- the id of the session to remove a blockblockId
- the id of an existing blockoldLocation
- the location of the sourcenewLocation
- the location of the destinationBlockDoesNotExistException
- if block id can not be foundBlockAlreadyExistsException
- if block id already exists in committed blocks of the
newLocationInvalidWorkerStateException
- if block id has not been committedWorkerOutOfSpaceException
- if newLocation does not have enough extra space to hold the
blockIOException
public void removeBlock(long sessionId, long blockId) throws InvalidWorkerStateException, BlockDoesNotExistException, IOException
BlockStore
removeBlock
in interface BlockStore
sessionId
- the id of the session to remove a blockblockId
- the id of an existing blockInvalidWorkerStateException
- if block id has not been committedBlockDoesNotExistException
- if block can not be foundIOException
public void removeBlock(long sessionId, long blockId, BlockStoreLocation location) throws InvalidWorkerStateException, BlockDoesNotExistException, IOException
BlockStore
removeBlock
in interface BlockStore
sessionId
- the id of the session to move a blockblockId
- the id of an existing blocklocation
- the location of the blockInvalidWorkerStateException
- if block id has not been committedBlockDoesNotExistException
- if block can not be foundIOException
public void accessBlock(long sessionId, long blockId) throws BlockDoesNotExistException
BlockStore
accessBlock
in interface BlockStore
sessionId
- the id of the session to access a blockblockId
- the id of an accessed blockBlockDoesNotExistException
- if the block id is not foundpublic void freeSpace(long sessionId, long availableBytes, BlockStoreLocation location) throws BlockDoesNotExistException, WorkerOutOfSpaceException, IOException
BlockStore
freeSpace
in interface BlockStore
sessionId
- the session idavailableBytes
- the amount of free space in byteslocation
- the location to free spaceBlockDoesNotExistException
- if blocks in EvictionPlan
can not be foundWorkerOutOfSpaceException
- if there is not enough spaceIOException
public void cleanupSession(long sessionId)
BlockStore
cleanupSession
in interface BlockStore
cleanupSession
in interface SessionCleanable
sessionId
- the session idpublic boolean hasBlockMeta(long blockId)
BlockStore
hasBlockMeta
in interface BlockStore
blockId
- the block idpublic BlockStoreMeta getBlockStoreMeta()
BlockStore
getBlockStoreMeta
in interface BlockStore
public BlockStoreMeta getBlockStoreMetaFull()
BlockStore
BlockStore.getBlockStoreMeta()
except that this includes
more information about the block store (e.g. blockId list). This is an expensive operation.getBlockStoreMetaFull
in interface BlockStore
public void registerBlockStoreEventListener(BlockStoreEventListener listener)
BlockStore
BlockStoreEventListener
to this block store.registerBlockStoreEventListener
in interface BlockStore
listener
- the listener to those eventspublic void updatePinnedInodes(Set<Long> inodes)
updatePinnedInodes
in interface BlockStore
inodes
- a set of ids inodes that are pinnedpublic boolean checkStorage()
BlockStore
checkStorage
in interface BlockStore
public void removeDir(StorageDir dir)
dir
- storage directory to be removedCopyright © 2023. All Rights Reserved.