alluxio.rocks

Class RocksStore

java.lang.Object

alluxio.rocks.RocksStore

All Implemented Interfaces:

Closeable, AutoCloseable

@NotThreadSafe
public final class RocksStore
extends Object
implements Closeable

Class for managing a rocksdb database. This class handles common functionality such as initializing the database and performing database backup/restore. This class provides locking methods for the callers. And the thread safety of RocksDB relies on the caller to use the corresponding lock methods. The reasons why this class only provides thread safety utilities to the callers (instead of wrapping it under each call) are: 1. Callers like RocksInodeStore and RocksBlockMetaStore have specific read/write logic like iteration, which cannot be abstracted and locked internally in this class. 2. With locking methods provided by this class, callers like RocksInodeStore can actually reuse the locks to perform concurrency control on their own logic. For reading/writing on the RocksDB, use the shared lock

   try (RocksSharedLockHandle r = mRocksStore.checkAndAcquireSharedLock() {
     // perform your read/write operation
   }
 

For operations like closing/restart/restoring on the RocksDB, an exclusive lock should be acquired by calling one of: 1. lockForClosing() 2. lockForRewrite() 3. lockForCheckpoint() Rule of thumb: 1. Use the proper locking methods when you access RocksDB. 2. Make each operation short. Make the locked section short. 3. If you have to make the operation long (like iteration), utilize shouldAbort(int) to check and abort voluntarily. See Javadoc on the locking methods for details.

Field Summary

Fields

Modifier and Type	Field and Description
LongAdder	mRefCount
AtomicStampedReference<Boolean>	mRocksDbStopServing
static java.time.Duration	ROCKS_CLOSE_WAIT_TIMEOUT
static int	ROCKS_OPEN_RETRY_TIMEOUT

Constructor Summary

Constructors

Constructor and Description
RocksStore(String name, String dbPath, String checkpointPath, org.rocksdb.DBOptions dbOpts, Collection<org.rocksdb.ColumnFamilyDescriptor> columnFamilyDescriptors, List<AtomicReference<org.rocksdb.ColumnFamilyHandle>> columnHandles, boolean reset)

Method Summary

Modifier and Type	Method and Description
RocksSharedLockHandle	checkAndAcquireSharedLock() This is the core logic of the shared lock mechanism.
static Optional<org.rocksdb.BlockBasedTableConfig>	checkSetTableConfig(PropertyKey cacheSize, PropertyKey bloomFilter, PropertyKey indexType, PropertyKey blockIndexType, List<org.rocksdb.RocksObject> toClose) loads RockDB configuration options based on property key configurations.
void	clear() Clears and re-initializes the database.
void	close()
org.rocksdb.RocksDB	getDb() Requires the caller to acquire a shared lock by calling checkAndAcquireSharedLock().
long	getSharedLockCount() Gets the number of shared lock on the RocksStore.
boolean	isServiceStopping() Checks whether the RocksDB is marked for exclusive access, so the operation should abort.
RocksExclusiveLockHandle	lockForCheckpoint() Before the process shuts down, acquire an exclusive lock on the RocksDB before closing.
RocksExclusiveLockHandle	lockForClosing() Before the process shuts down, acquire an exclusive lock on the RocksDB before closing.
RocksExclusiveLockHandle	lockForRewrite() Before the process shuts down, acquire an exclusive lock on the RocksDB before closing.
void	restoreFromCheckpoint(CheckpointInputStream input) Restores the database from a checkpoint.
void	restoreFromCheckpoint(File directory) Restores RocksDB state from a checkpoint at the provided location.
void	shouldAbort(int lockedVersion) Used by ongoing r/w operations to check if the operation needs to abort and yield to the RocksDB shutdown.
void	writeToCheckpoint(File directory) Writes a checkpoint under the specified directory.
void	writeToCheckpoint(OutputStream output) Writes a checkpoint of the database's content to the given output stream.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

ROCKS_OPEN_RETRY_TIMEOUT

public static final int ROCKS_OPEN_RETRY_TIMEOUT

See Also:

Constant Field Values

ROCKS_CLOSE_WAIT_TIMEOUT

public static final java.time.Duration ROCKS_CLOSE_WAIT_TIMEOUT

mRocksDbStopServing

public final AtomicStampedReference<Boolean> mRocksDbStopServing

mRefCount

public volatile LongAdder mRefCount

Constructor Detail

RocksStore

public RocksStore(String name,
                  String dbPath,
                  String checkpointPath,
                  org.rocksdb.DBOptions dbOpts,
                  Collection<org.rocksdb.ColumnFamilyDescriptor> columnFamilyDescriptors,
                  List<AtomicReference<org.rocksdb.ColumnFamilyHandle>> columnHandles,
                  boolean reset)

Parameters:

name - a name to distinguish what store this is

dbPath - a path for the rocks database

checkpointPath - a path for taking database checkpoints

dbOpts - the configured RocksDB options

columnFamilyDescriptors - columns to create within the rocks database

columnHandles - column handle references to populate

reset - if true, the RocksDB will be cleaned up and then new DB will be created

Method Detail

getDb

public org.rocksdb.RocksDB getDb()

Requires the caller to acquire a shared lock by calling checkAndAcquireSharedLock().

Returns:

the underlying rocksdb instance. The instance changes when clear() is called, so if the caller caches the returned db, they must reset it after calling clear()

clear

public void clear()

Clears and re-initializes the database. Requires the caller to acquire exclusive lock by calling lockForRewrite().

writeToCheckpoint

public void writeToCheckpoint(File directory)
                       throws org.rocksdb.RocksDBException

Writes a checkpoint under the specified directory.

Parameters:

directory - that the checkpoint will be written under

Throws:

org.rocksdb.RocksDBException - if it encounters and error when writing the checkpoint

writeToCheckpoint

public void writeToCheckpoint(OutputStream output)
                       throws IOException,
                              InterruptedException

Writes a checkpoint of the database's content to the given output stream. Requires the caller to acquire an exclusive lock by calling lockForCheckpoint().

Parameters:

output - the stream to write to

Throws:

IOException

InterruptedException

restoreFromCheckpoint

public void restoreFromCheckpoint(File directory)
                           throws org.rocksdb.RocksDBException,
                                  IOException

Restores RocksDB state from a checkpoint at the provided location. Moves the directory to a permanent location, restores RocksDB state, and then immediately takes a new snapshot in the original location as replacement.

Parameters:

directory - where the checkpoint is located

Throws:

org.rocksdb.RocksDBException - if rocks encounters a problem

IOException - if moving files around encounters a problem

restoreFromCheckpoint

public void restoreFromCheckpoint(CheckpointInputStream input)
                           throws IOException

Restores the database from a checkpoint. Requires the caller to acquire an exclusive lock by calling lockForRewrite().

Parameters:

input - the checkpoint stream to restore from

Throws:

IOException

close

public void close()

Specified by:

close in interface Closeable

Specified by:

close in interface AutoCloseable

checkSetTableConfig

public static Optional<org.rocksdb.BlockBasedTableConfig> checkSetTableConfig(PropertyKey cacheSize,
                                                                              PropertyKey bloomFilter,
                                                                              PropertyKey indexType,
                                                                              PropertyKey blockIndexType,
                                                                              List<org.rocksdb.RocksObject> toClose)

loads RockDB configuration options based on property key configurations.

Parameters:

cacheSize - integer PropertyKey for cache size

bloomFilter - boolean PropertyKey for bloom filter

indexType - enum PropertyKey for index type

blockIndexType - enum PropertyKey for block index type

toClose - the objects to be closed

Returns:

an Optional BlockBasedTableConfig

checkAndAcquireSharedLock

public RocksSharedLockHandle checkAndAcquireSharedLock()

This is the core logic of the shared lock mechanism. Before any r/w operation on the RocksDB, acquire a shared lock with this method. The shared lock guarantees the RocksDB will not be restarted/cleared during the r/w access. In other words, similar to a read-write lock, exclusive lock requests will wait for shared locks to be released first. However, note that exclusive lock acquisition only waits for a certain period of time, defined by PropertyKey.MASTER_METASTORE_ROCKS_EXCLUSIVE_LOCK_TIMEOUT. After this timeout, the exclusive lock will be forced, and the shared lock holders are disrespected. Normally, the r/w operation should either complete or abort within seconds so the timeout PropertyKey.MASTER_METASTORE_ROCKS_EXCLUSIVE_LOCK_TIMEOUT should not be exceeded at all.

Returns:

a shared lock handle used to manage and close the shared lock

lockForClosing

public RocksExclusiveLockHandle lockForClosing()

Before the process shuts down, acquire an exclusive lock on the RocksDB before closing. Note this lock only exists on the Alluxio side. A STOP_SERVING flag will be set so all existing readers/writers will abort asap. The exclusive lock ensures there are no existing concurrent r/w operations, so it is safe to close the RocksDB and recycle all relevant resources. The STOP_SERVING status will NOT be reset, because the process will shut down soon.

Returns:

the exclusive lock handle used to manage and close the lock

lockForCheckpoint

public RocksExclusiveLockHandle lockForCheckpoint()

Before the process shuts down, acquire an exclusive lock on the RocksDB before closing. Note this lock only exists on the Alluxio side. A STOP_SERVING flag will be set so all existing readers/writers will abort asap. The exclusive lock ensures there are no existing concurrent r/w operations, so it is safe to restart/checkpoint the RocksDB and update the DB reference. The STOP_SERVING status will be reset and the RocksDB will be open for operations again. The version will not be bumped up, because the RocksDB contents has not changed. See checkAndAcquireSharedLock() for how this affects the shared lock logic.

Returns:

the exclusive lock handle used to manage and close the lock

lockForRewrite

public RocksExclusiveLockHandle lockForRewrite()

Before the process shuts down, acquire an exclusive lock on the RocksDB before closing. Note this lock only exists on the Alluxio side. A STOP_SERVING flag will be set so all existing readers/writers will abort asap. The exclusive lock ensures there are no existing concurrent r/w operations, so it is safe to restart/checkpoint the RocksDB and update the DB reference. The STOP_SERVING status will be reset and the RocksDB will be open for operations again. The version will be bumped up, because the RocksDB contents has changed. If there is one slow operation expecting to see the old version, that operation should abort. See checkAndAcquireSharedLock() for how this affects the shared lock logic.

Returns:

the exclusive lock handle used to manage and close the lock

shouldAbort

public void shouldAbort(int lockedVersion)

Used by ongoing r/w operations to check if the operation needs to abort and yield to the RocksDB shutdown.

Parameters:

lockedVersion - The RocksDB version from the shared lock. This version is used to tell if a restore or clear operation has happened on the RocksDB.

isServiceStopping

public boolean isServiceStopping()

Checks whether the RocksDB is marked for exclusive access, so the operation should abort.

Returns:

whether the RocksDB expects to stop

getSharedLockCount

public long getSharedLockCount()

Gets the number of shared lock on the RocksStore.

Returns:

the count