alluxio.worker.block

Interface BlockWorker

All Superinterfaces:

DataWorker, Server<WorkerNetAddress>, SessionCleanable, Worker

public interface BlockWorker
extends DataWorker, SessionCleanable

A block worker in the Alluxio system.

Method Summary

Modifier and Type	Method and Description
void	abortBlock(long sessionId, long blockId) Aborts the temporary block created by the session.
void	accessBlock(long sessionId, long blockId) Access the block for a given session.
void	asyncCache(AsyncCacheRequest request) Deprecated. This method will be deprecated as of v3.0, use cache(alluxio.grpc.CacheRequest)
void	cache(CacheRequest request) Submits the cache request to cache manager to execute.
void	clearMetrics() Clears the worker metrics.
void	closeUfsBlock(long sessionId, long blockId) Closes a UFS block for a client session.
void	commitBlock(long sessionId, long blockId, boolean pinOnCreate) Commits a block to Alluxio managed space.
String	createBlock(long sessionId, long blockId, int tier, CreateBlockOptions createBlockOptions) Creates a block in Alluxio managed space.
BlockReader	createBlockReader(long sessionId, long blockId, long offset, boolean positionShort, Protocol.OpenUfsBlockOptions options) Creates the block reader to read from Alluxio block or UFS block.
void	createBlockRemote(long sessionId, long blockId, String tierAlias, long initialBytes) Creates a block.
BlockWriter	createBlockWriter(long sessionId, long blockId) Creates a BlockWriter for an existing temporary block which is already created by createBlock(long, long, int, alluxio.worker.block.CreateBlockOptions).
BlockReader	createUfsBlockReader(long sessionId, long blockId, long offset, boolean positionShort, Protocol.OpenUfsBlockOptions options) Creates a block reader to read a UFS block starting from given block offset.
void	freeWorker() Frees all blocks in the current worker by deleting all block store directories.
BlockStore	getBlockStore()
Configuration	getConfiguration(GetConfigurationPOptions options)
FileInfo	getFileInfo(long fileId) Gets the file information.
BlockHeartbeatReport	getReport() Gets a report for the periodic heartbeat to master.
BlockStoreMeta	getStoreMeta() Gets the metadata for the entire block store.
BlockStoreMeta	getStoreMetaFull() Similar as getStoreMeta() except that this also contains full blockId list.
List<String>	getWhiteList()
WorkerNetAddress	getWorkerAddress()
CompletableFuture<List<BlockStatus>>	load(List<Block> fileBlocks, UfsReadOptions options) Load blocks into alluxio.
long	lockBlock(long sessionId, long blockId) Obtains a read lock the block.
long	lockBlockNoException(long sessionId, long blockId) Obtains a read lock the block without throwing an exception.
void	moveBlock(long sessionId, long blockId, String tierAlias) Moves a block from its current location to a target location, currently only tier level moves are supported.
boolean	openUfsBlock(long sessionId, long blockId, Protocol.OpenUfsBlockOptions options) Opens a UFS block.
String	readBlock(long sessionId, long blockId, long lockId) Gets the path to the block file in local storage.
BlockReader	readBlockRemote(long sessionId, long blockId, long lockId) Gets the block reader for the block.
BlockReader	readUfsBlock(long sessionId, long blockId, long offset) Gets a block reader to read a UFS block.
void	removeBlock(long sessionId, long blockId) Frees a block from Alluxio managed space.
void	requestSpace(long sessionId, long blockId, long additionalBytes) Request an amount of space for a block in its storage directory.
void	unlockBlock(long lockId) Releases the lock with the specified lock id.
boolean	unlockBlock(long sessionId, long blockId) Releases the lock with the specified session and block id.
void	updatePinList(Set<Long> pinnedInodes) Sets the pinlist for the underlying block store.

Methods inherited from interface alluxio.worker.DataWorker

getWorkerId

Methods inherited from interface alluxio.Server

close, getDependencies, getName, getServices, start, stop

Methods inherited from interface alluxio.worker.SessionCleanable

cleanupSession

Method Detail

abortBlock

void abortBlock(long sessionId,
                long blockId)
         throws IOException

Aborts the temporary block created by the session.

Parameters:

sessionId - the id of the client

blockId - the id of the block to be aborted

Throws:

IOException

commitBlock

void commitBlock(long sessionId,
                 long blockId,
                 boolean pinOnCreate)

Commits a block to Alluxio managed space. The block must be temporary. The block will not be persisted or accessible before commitBlock succeeds.

Parameters:

sessionId - the id of the client

blockId - the id of the block to commit

pinOnCreate - whether to pin block on create

createBlock

String createBlock(long sessionId,
                   long blockId,
                   int tier,
                   CreateBlockOptions createBlockOptions)

Creates a block in Alluxio managed space. Calls createBlockWriter(long, long) to get a writer for writing to the block. The block will be temporary until it is committed by commitBlock(long, long, boolean) . Throws an IllegalArgumentException if the location does not belong to tiered storage.

Parameters:

sessionId - the id of the client

blockId - the id of the block to create

tier - the tier to place the new block in BlockStoreLocation.ANY_TIER for any tier

createBlockOptions - the createBlockOptions

Returns:

a string representing the path to the local file

createBlockWriter

BlockWriter createBlockWriter(long sessionId,
                              long blockId)
                       throws IOException

Creates a BlockWriter for an existing temporary block which is already created by createBlock(long, long, int, alluxio.worker.block.CreateBlockOptions).

Parameters:

sessionId - the id of the client

blockId - the id of the block to be opened for writing

Returns:

the block writer for the local block file

Throws:

IOException

getReport

BlockHeartbeatReport getReport()

Gets a report for the periodic heartbeat to master. Contains the blocks added since the last heart beat and blocks removed since the last heartbeat.

Returns:

a block heartbeat report

getStoreMeta

BlockStoreMeta getStoreMeta()

Gets the metadata for the entire block store. Contains the block mapping per storage dir and the total capacity and used capacity of each tier. This function is cheap.

Returns:

the block store metadata

getStoreMetaFull

BlockStoreMeta getStoreMetaFull()

Similar as getStoreMeta() except that this also contains full blockId list. This function is expensive.

Returns:

the full block store metadata

createBlockReader

BlockReader createBlockReader(long sessionId,
                              long blockId,
                              long offset,
                              boolean positionShort,
                              Protocol.OpenUfsBlockOptions options)
                       throws IOException

Creates the block reader to read from Alluxio block or UFS block. Owner of this block reader must close it or lock will leak.

Parameters:

sessionId - the client session ID

blockId - the ID of the UFS block to read

offset - the offset within the block

positionShort - whether the operation is using positioned read to a small buffer size

options - the options

Returns:

a block reader to read data from

Throws:

IOException - if it fails to get block reader

createUfsBlockReader

BlockReader createUfsBlockReader(long sessionId,
                                 long blockId,
                                 long offset,
                                 boolean positionShort,
                                 Protocol.OpenUfsBlockOptions options)
                          throws IOException

Creates a block reader to read a UFS block starting from given block offset. Owner of this block reader must close it to cleanup state.

Parameters:

sessionId - the client session ID

blockId - the ID of the UFS block to read

offset - the offset within the block

positionShort - whether the operation is using positioned read to a small buffer size

options - the options

Returns:

the block reader instance

Throws:

IOException - if it fails to get block reader

removeBlock

void removeBlock(long sessionId,
                 long blockId)
          throws IOException

Frees a block from Alluxio managed space.

Parameters:

sessionId - the id of the client

blockId - the id of the block to be freed

Throws:

IOException

freeWorker

void freeWorker()
         throws IOException

Frees all blocks in the current worker by deleting all block store directories. Whether this method returns successfully or exceptionally, the worker should not be used in any way.

Throws:

IOException - if free fails

requestSpace

void requestSpace(long sessionId,
                  long blockId,
                  long additionalBytes)

Request an amount of space for a block in its storage directory. The block must be a temporary block.

Parameters:

sessionId - the id of the client

blockId - the id of the block to allocate space to

additionalBytes - the amount of bytes to allocate

asyncCache

@Deprecated
void asyncCache(AsyncCacheRequest request)

Deprecated. This method will be deprecated as of v3.0, use cache(alluxio.grpc.CacheRequest)

Submits the async cache request to async cache manager to execute.

Parameters:

request - the async cache request

cache

void cache(CacheRequest request)
    throws AlluxioException,
           IOException

Submits the cache request to cache manager to execute.

Parameters:

request - the cache request

Throws:

AlluxioException

IOException

load

CompletableFuture<List<BlockStatus>> load(List<Block> fileBlocks,
                                          UfsReadOptions options)

Load blocks into alluxio.

Parameters:

fileBlocks - list of fileBlocks, one file blocks contains blocks belong to one file

options - read ufs options

Returns:

future of load status for failed blocks

updatePinList

void updatePinList(Set<Long> pinnedInodes)

Sets the pinlist for the underlying block store.

Parameters:

pinnedInodes - a set of pinned inodes

getFileInfo

FileInfo getFileInfo(long fileId)
              throws IOException

Gets the file information.

Parameters:

fileId - the file id

Returns:

the file info

Throws:

IOException

clearMetrics

void clearMetrics()

Clears the worker metrics.

getConfiguration

Configuration getConfiguration(GetConfigurationPOptions options)

Parameters:

options - method options

Returns:

configuration information list

getWhiteList

List<String> getWhiteList()

Returns:

the white list

getBlockStore

BlockStore getBlockStore()

Returns:

the block store

getWorkerAddress

WorkerNetAddress getWorkerAddress()

Returns:

the worker address

readBlockRemote

BlockReader readBlockRemote(long sessionId,
                            long blockId,
                            long lockId)
                     throws BlockDoesNotExistException,
                            InvalidWorkerStateException,
                            IOException

Gets the block reader for the block. This method is only called by a data server.

Parameters:

sessionId - the id of the client

blockId - the id of the block to read

lockId - the id of the lock on this block

Returns:

the block reader for the block

Throws:

BlockDoesNotExistException - if lockId is not found

InvalidWorkerStateException - if sessionId or blockId is not the same as that in the LockRecord of lockId

IOException

openUfsBlock

boolean openUfsBlock(long sessionId,
                     long blockId,
                     Protocol.OpenUfsBlockOptions options)
              throws BlockAlreadyExistsException

Opens a UFS block. It registers the block metadata information to the UFS block store. It throws an UfsBlockAccessTokenUnavailableException if the number of concurrent readers on this block exceeds a threshold.

Parameters:

sessionId - the session ID

blockId - the block ID

options - the options

Returns:

whether the UFS block is successfully opened

Throws:

BlockAlreadyExistsException - if the UFS block already exists in the UnderFileSystemBlockStore

closeUfsBlock

void closeUfsBlock(long sessionId,
                   long blockId)
            throws BlockAlreadyExistsException,
                   BlockDoesNotExistException,
                   IOException,
                   WorkerOutOfSpaceException

Closes a UFS block for a client session. It also commits the block to Alluxio block store if the UFS block has been cached successfully.

Parameters:

sessionId - the session ID

blockId - the block ID

Throws:

BlockAlreadyExistsException - if it fails to commit the block to Alluxio block store because the block exists in the Alluxio block store

BlockDoesNotExistException - if the UFS block does not exist in the UnderFileSystemBlockStore

WorkerOutOfSpaceException - the the worker does not have enough space to commit the block

IOException

accessBlock

void accessBlock(long sessionId,
                 long blockId)
          throws BlockDoesNotExistException

Access the block for a given session. This should be called to update the evictor when necessary.

Parameters:

sessionId - the id of the client

blockId - the id of the block to access

Throws:

BlockDoesNotExistException - this exception is not thrown in the tiered block store implementation

moveBlock

void moveBlock(long sessionId,
               long blockId,
               String tierAlias)
        throws BlockDoesNotExistException,
               BlockAlreadyExistsException,
               InvalidWorkerStateException,
               WorkerOutOfSpaceException,
               IOException

Moves a block from its current location to a target location, currently only tier level moves are supported. Throws an IllegalArgumentException if the tierAlias is out of range of tiered storage.

Parameters:

sessionId - the id of the client

blockId - the id of the block to move

tierAlias - the alias of the tier to move the block to

Throws:

BlockDoesNotExistException - if blockId cannot be found

BlockAlreadyExistsException - if blockId already exists in committed blocks of the newLocation

InvalidWorkerStateException - if blockId has not been committed

WorkerOutOfSpaceException - if newLocation does not have enough extra space to hold the block

IOException

readBlock

String readBlock(long sessionId,
                 long blockId,
                 long lockId)
          throws BlockDoesNotExistException,
                 InvalidWorkerStateException

Gets the path to the block file in local storage. The block must be a permanent block, and the caller must first obtain the lock on the block.

Parameters:

sessionId - the id of the client

blockId - the id of the block to read

lockId - the id of the lock on this block

Returns:

a string representing the path to this block in local storage

Throws:

BlockDoesNotExistException - if the blockId cannot be found in committed blocks or lockId cannot be found

InvalidWorkerStateException - if sessionId or blockId is not the same as that in the LockRecord of lockId

unlockBlock

void unlockBlock(long lockId)
          throws BlockDoesNotExistException

Releases the lock with the specified lock id.

Parameters:

lockId - the id of the lock to release

Throws:

BlockDoesNotExistException - if lock id cannot be found

unlockBlock

boolean unlockBlock(long sessionId,
                    long blockId)

Releases the lock with the specified session and block id.

Parameters:

sessionId - the session id

blockId - the block id

Returns:

false if it fails to unlock due to the lock is not found

lockBlock

long lockBlock(long sessionId,
               long blockId)
        throws BlockDoesNotExistException

Obtains a read lock the block.

Parameters:

sessionId - the id of the client

blockId - the id of the block to be locked

Returns:

the lock id that uniquely identifies the lock obtained

Throws:

BlockDoesNotExistException - if blockId cannot be found, for example, evicted already

lockBlockNoException

long lockBlockNoException(long sessionId,
                          long blockId)

Obtains a read lock the block without throwing an exception. If the lock fails, return BlockLockManager#INVALID_LOCK_ID.

Parameters:

sessionId - the id of the client

blockId - the id of the block to be locked

Returns:

the lock id that uniquely identifies the lock obtained or BlockLockManager#INVALID_LOCK_ID if it failed to lock

createBlockRemote

void createBlockRemote(long sessionId,
                       long blockId,
                       String tierAlias,
                       long initialBytes)
                throws BlockAlreadyExistsException,
                       WorkerOutOfSpaceException,
                       IOException

Creates a block. This method is only called from a data server. Calls #getTempBlockWriterRemote(long, long) to get a writer for writing to the block. Throws an IllegalArgumentException if the location doens not belong to tiered storage.

Parameters:

sessionId - the id of the client

blockId - the id of the block to be created

tierAlias - the alias of the tier to place the new block in

initialBytes - the initial amount of bytes to be allocated

Throws:

BlockAlreadyExistsException - if blockId already exists, either temporary or committed, or block in eviction plan already exists

WorkerOutOfSpaceException - if this Store has no more space than the initialBlockSize

IOException

readUfsBlock

BlockReader readUfsBlock(long sessionId,
                         long blockId,
                         long offset)
                  throws BlockDoesNotExistException,
                         IOException

Gets a block reader to read a UFS block. This method is only called by the data server.

Parameters:

sessionId - the client session ID

blockId - the ID of the UFS block to read

offset - the offset within the block

Returns:

the block reader instance

Throws:

BlockDoesNotExistException - if the block does not exist in the UFS block store

IOException