alluxio.master.file.meta

Class InodeTree

java.lang.Object

alluxio.master.file.meta.InodeTree

All Implemented Interfaces:

Checkpointed, DelegatingJournaled, Journaled, JournalEntryIterable

@NotThreadSafe
public class InodeTree
extends Object
implements DelegatingJournaled

Represents the tree of Inodes.

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static class	InodeTree.LockPattern Patterns of inode path locking.

Field Summary

Fields

Modifier and Type	Field and Description
static long	NO_PARENT Value to be used for an inode with no parent.
static int	PATH_TRAVERSAL_RETRIES Number of retries when trying to lock a path, from a given id.
static String	ROOT_INODE_NAME Only the root inode should have the empty string as its name.
static String	ROOT_PATH Path of the root inode.

Constructor Summary

Constructors

Constructor and Description
InodeTree(InodeStore inodeStore, ContainerIdGenerable containerIdGenerator, InodeDirectoryIdGenerator directoryIdGenerator, MountTable mountTable, InodeLockManager lockManager)

Method Summary

Modifier and Type	Method and Description
void	cacheOperation(OperationId opId) Used to mark an operation as complete in retry-cache.
void	close() Close resources associated with this tree instance.
List<Inode>	createPath(RpcContext rpcContext, LockedInodePath inodePath, CreatePathContext<?,?> context) Creates a file or directory at path.
void	deleteInode(RpcContext rpcContext, LockedInodePath inodePath, long opTimeMs) Deletes a single inode from the inode tree by removing it from the parent inode.
void	ensureFullInodePath(LockedInodePath inodePath) Attempts to extend an existing LockedInodePath to reach the target inode (the last inode for the full path).
Journaled	getDelegate()
LockedInodePathList	getDescendants(LockedInodePath inodePath) Returns LockedInodePaths for all descendants of inodePath.
Map<Long,Number>	getFileSizeHistogram()
long	getInodeCount()
InodeLockManager	getInodeLockManager()
AlluxioURI	getPath(InodeView inode) Returns the path for a particular inode.
AlluxioURI	getPath(long id) Returns the path for a particular inode id.
ArrayList<String>	getPathInodeNames(InodeView inode) Computes the path components for the given inode view.
ArrayList<String>	getPathInodeNames(long id) Computes the path components for the given inode id.
Set<Long>	getPinIdSet()
int	getPinnedSize()
Set<Long>	getReplicationLimitedFileIds()
InodeDirectory	getRoot()
String	getRootUserName()
Set<Long>	getToBePersistedIds()
TtlBucketList	getTtlBuckets()
void	initializeRoot(String owner, String group, Mode mode, JournalContext context) Initializes the root of the inode tree.
boolean	inodeIdExists(long id)
boolean	inodePathExists(AlluxioURI uri)
boolean	isOperationComplete(OperationId opId) Whether given operation is still cached in retry-cache.
boolean	isRootId(long fileId)
LockedInodePath	lockFullInodePath(AlluxioURI uri, InodeTree.LockPattern lockPattern, JournalContext journalContext) Locks a path and throws an exception if the path does not exist.
LockedInodePath	lockFullInodePath(AlluxioURI uri, LockingScheme lockScheme, JournalContext journalContext) Locks a path and throws an exception if the path does not exist.
LockedInodePath	lockFullInodePath(long id, InodeTree.LockPattern lockPattern, JournalContext journalContext) Locks a path and throws an exception if the path does not exist.
LockedInodePath	lockInodePath(AlluxioURI uri, InodeTree.LockPattern lockPattern, boolean tryLock, JournalContext journalContext) Locks existing inodes on the specified path, in the specified InodeTree.LockPattern.
LockedInodePath	lockInodePath(AlluxioURI uri, InodeTree.LockPattern lockPattern, JournalContext journalContext) Locks existing inodes on the specified path, in the specified InodeTree.LockPattern.
LockedInodePath	lockInodePath(LockingScheme scheme, JournalContext journalContext) Locks existing inodes on the path and with the pattern defined by LockingScheme.getPath() and LockingScheme.getPattern().
InodePathPair	lockInodePathPair(AlluxioURI path1, InodeTree.LockPattern lockPattern1, AlluxioURI path2, InodeTree.LockPattern lockPattern2, JournalContext journalContext) Locks existing inodes on the two specified paths.
long	newBlock(java.util.function.Supplier<JournalContext> context, File.NewBlockEntry entry)
void	rename(java.util.function.Supplier<JournalContext> context, File.RenameEntry entry)
void	setAcl(java.util.function.Supplier<JournalContext> context, File.SetAclEntry entry)
void	setDirectChildrenLoaded(java.util.function.Supplier<JournalContext> context, InodeDirectory dir) Marks an inode directory as having its direct children loaded.
void	setDirectChildrenLoaded(java.util.function.Supplier<JournalContext> context, InodeDirectory dir, boolean directChildrenLoaded) Marks an inode directory as having its direct children loaded or not.
void	setPinned(RpcContext rpcContext, LockedInodePath inodePath, boolean pinned, List<String> mediumTypes, long opTimeMs) Sets the pinned state of an inode.
void	setReplication(RpcContext rpcContext, LockedInodePath inodePath, Integer replicationMax, Integer replicationMin, long opTimeMs) Sets the min and/or max replication level of an inode.
void	syncPersistExistingDirectory(java.util.function.Supplier<JournalContext> context, InodeDirectoryView dir, boolean isMetadataLoad) Synchronously persists an inode directory to the UFS.
void	syncPersistNewDirectory(MutableInodeDirectory dir, CreatePathContext<?,?> createContext) Synchronously persists an inode directory to the UFS.
LockedInodePath	tryLockInodePath(LockingScheme scheme, JournalContext journalContext) Locks existing inodes on the path and with the pattern defined by LockingScheme.getPath() and LockingScheme.getPattern().
void	updateInode(java.util.function.Supplier<JournalContext> context, File.UpdateInodeEntry entry)
File.UpdateInodeEntry	updateInodeAccessTimeNoJournal(long inodeId, long accessTime)
void	updateInodeFile(java.util.function.Supplier<JournalContext> context, File.UpdateInodeFileEntry entry)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface alluxio.master.journal.DelegatingJournaled

applyAndJournal, getCheckpointName, getJournalEntryIterator, processJournalEntry, resetState, restoreFromCheckpoint, restoreFromCheckpoint, writeToCheckpoint, writeToCheckpoint

Field Detail

NO_PARENT

public static final long NO_PARENT

Value to be used for an inode with no parent.

See Also:

Constant Field Values

ROOT_INODE_NAME

public static final String ROOT_INODE_NAME

Only the root inode should have the empty string as its name.

See Also:

Constant Field Values

ROOT_PATH

public static final String ROOT_PATH

Path of the root inode.

See Also:

Constant Field Values

PATH_TRAVERSAL_RETRIES

public static final int PATH_TRAVERSAL_RETRIES

Number of retries when trying to lock a path, from a given id.

See Also:

Constant Field Values

Constructor Detail

InodeTree

public InodeTree(InodeStore inodeStore,
                 ContainerIdGenerable containerIdGenerator,
                 InodeDirectoryIdGenerator directoryIdGenerator,
                 MountTable mountTable,
                 InodeLockManager lockManager)

Parameters:

inodeStore - the inode store

containerIdGenerator - the container id generator to use to get new container ids

directoryIdGenerator - the directory id generator to use to get new directory ids

mountTable - the mount table to manage the file system mount points

lockManager - inode lock manager

Method Detail

isOperationComplete

public boolean isOperationComplete(@Nullable
                                   OperationId opId)

Whether given operation is still cached in retry-cache.

Parameters:

opId - the operation id

Returns:

true if given op is marked complete

cacheOperation

public void cacheOperation(@Nullable
                           OperationId opId)

Used to mark an operation as complete in retry-cache.

Parameters:

opId - the operation id

initializeRoot

public void initializeRoot(String owner,
                           String group,
                           Mode mode,
                           JournalContext context)
                    throws UnavailableException

Initializes the root of the inode tree.

Parameters:

owner - the root owner

group - the root group

mode - the root mode

context - the journal context to journal the initialization to

Throws:

UnavailableException

getTtlBuckets

public TtlBucketList getTtlBuckets()

Returns:

the list of TTL buckets for tracking inode TTLs

getInodeCount

public long getInodeCount()

Returns:

the number of inodes in the inode tree

getFileSizeHistogram

public Map<Long,Number> getFileSizeHistogram()

Returns:

the file size histogram of the inode tree

setDirectChildrenLoaded

public void setDirectChildrenLoaded(java.util.function.Supplier<JournalContext> context,
                                    InodeDirectory dir)

Marks an inode directory as having its direct children loaded.

Parameters:

context - journal context supplier

dir - the inode directory

setDirectChildrenLoaded

public void setDirectChildrenLoaded(java.util.function.Supplier<JournalContext> context,
                                    InodeDirectory dir,
                                    boolean directChildrenLoaded)

Marks an inode directory as having its direct children loaded or not.

Parameters:

context - journal context supplier

dir - the inode directory

directChildrenLoaded - whether to load the direct children if they were not loaded before

newBlock

public long newBlock(java.util.function.Supplier<JournalContext> context,
                     File.NewBlockEntry entry)

Parameters:

context - journal context supplier

entry - an entry representing an add block operation

Returns:

the new block id

updateInodeFile

public void updateInodeFile(java.util.function.Supplier<JournalContext> context,
                            File.UpdateInodeFileEntry entry)

Parameters:

context - journal context supplier

entry - an entry representing an update inode file operation

updateInode

public void updateInode(java.util.function.Supplier<JournalContext> context,
                        File.UpdateInodeEntry entry)

Parameters:

context - journal context supplier

entry - an entry representing an update inode operation

updateInodeAccessTimeNoJournal

public File.UpdateInodeEntry updateInodeAccessTimeNoJournal(long inodeId,
                                                            long accessTime)

Parameters:

inodeId - id of inode to be updated

accessTime - the new access time

Returns:

the applied inode entry

rename

public void rename(java.util.function.Supplier<JournalContext> context,
                   File.RenameEntry entry)

Parameters:

context - journal context supplier

entry - an entry representing a rename operation

setAcl

public void setAcl(java.util.function.Supplier<JournalContext> context,
                   File.SetAclEntry entry)

Parameters:

context - journal context supplier

entry - an entry representing a set acl operation

getRootUserName

@Nullable
public String getRootUserName()

Returns:

username of root of inode tree, null if the inode tree is not initialized

getPinnedSize

public int getPinnedSize()

Returns:

the number of pinned inodes

inodeIdExists

public boolean inodeIdExists(long id)

Parameters:

id - the id to get the inode for

Returns:

whether the inode exists

tryLockInodePath

public LockedInodePath tryLockInodePath(LockingScheme scheme,
                                        JournalContext journalContext)
                                 throws InvalidPathException

Locks existing inodes on the path and with the pattern defined by LockingScheme.getPath() and LockingScheme.getPattern(). This method uses the Lock.tryLock() method to gain ownership of the locks. The reason one might want to use this is to avoid the fairness heuristics within the ReentrantReadWriteLock's NonFairSync which may block reader threads if a writer if the first in the queue.

Parameters:

scheme - the locking scheme to lock the path with

journalContext - the journal context to flush when the LockedInodePath releases

Returns:

the LockedInodePath representing the locked path of inodes

Throws:

InvalidPathException - if the path is invalid

lockInodePath

public LockedInodePath lockInodePath(LockingScheme scheme,
                                     JournalContext journalContext)
                              throws InvalidPathException

Locks existing inodes on the path and with the pattern defined by LockingScheme.getPath() and LockingScheme.getPattern().

Parameters:

scheme - the locking scheme to lock the path with

journalContext - the journal context to flush when the LockedInodePath releases

Returns:

the LockedInodePath representing the locked path of inodes

Throws:

InvalidPathException - if the path is invalid

lockInodePath

public LockedInodePath lockInodePath(AlluxioURI uri,
                                     InodeTree.LockPattern lockPattern,
                                     JournalContext journalContext)
                              throws InvalidPathException

Locks existing inodes on the specified path, in the specified InodeTree.LockPattern. The target inode is not required to exist.

Parameters:

uri - the uri to lock

lockPattern - the InodeTree.LockPattern to lock the inodes with

journalContext - the journal context to flush when the LockedInodePath releases

Returns:

the LockedInodePath representing the locked path of inodes

Throws:

InvalidPathException - if the path is invalid

lockInodePath

public LockedInodePath lockInodePath(AlluxioURI uri,
                                     InodeTree.LockPattern lockPattern,
                                     boolean tryLock,
                                     JournalContext journalContext)
                              throws InvalidPathException

Locks existing inodes on the specified path, in the specified InodeTree.LockPattern. The target inode is not required to exist.

Parameters:

uri - the uri to lock

lockPattern - the InodeTree.LockPattern to lock the inodes with

tryLock - true to use Lock.tryLock() or false to use Lock.lock()

journalContext - the journal context to flush when the LockedInodePath releases

Returns:

the LockedInodePath representing the locked path of inodes

Throws:

InvalidPathException - if the path is invalid

inodePathExists

public boolean inodePathExists(AlluxioURI uri)

Parameters:

uri - the AlluxioURI to check for existence

Returns:

whether the inode exists

lockFullInodePath

public LockedInodePath lockFullInodePath(AlluxioURI uri,
                                         LockingScheme lockScheme,
                                         JournalContext journalContext)
                                  throws InvalidPathException,
                                         FileDoesNotExistException

Locks a path and throws an exception if the path does not exist.

Parameters:

uri - a uri to lock

lockScheme - the scheme to lock with

journalContext - the journal context to flush when the LockedInodePath releases

Returns:

a locked inode path for the uri

Throws:

InvalidPathException

FileDoesNotExistException

lockFullInodePath

public LockedInodePath lockFullInodePath(AlluxioURI uri,
                                         InodeTree.LockPattern lockPattern,
                                         JournalContext journalContext)
                                  throws InvalidPathException,
                                         FileDoesNotExistException

Locks a path and throws an exception if the path does not exist.

Parameters:

uri - a uri to lock

lockPattern - the pattern to lock with

journalContext - the journal context to flush when the LockedInodePath releases

Returns:

a locked inode path for the uri

Throws:

InvalidPathException

FileDoesNotExistException

lockFullInodePath

public LockedInodePath lockFullInodePath(long id,
                                         InodeTree.LockPattern lockPattern,
                                         JournalContext journalContext)
                                  throws FileDoesNotExistException

Locks a path and throws an exception if the path does not exist.

Parameters:

id - the inode id to lock

lockPattern - the pattern to lock with

journalContext - the journal context to flush when the LockedInodePath releases

Returns:

a locked inode path for the uri

Throws:

FileDoesNotExistException

lockInodePathPair

public InodePathPair lockInodePathPair(AlluxioURI path1,
                                       InodeTree.LockPattern lockPattern1,
                                       AlluxioURI path2,
                                       InodeTree.LockPattern lockPattern2,
                                       JournalContext journalContext)
                                throws InvalidPathException

Locks existing inodes on the two specified paths. The two paths will be locked in the correct order. The target inodes are not required to exist.

Parameters:

path1 - the first path to lock

lockPattern1 - the locking pattern for the first path

path2 - the second path to lock

lockPattern2 - the locking pattern for the second path

journalContext - the journal context to flush when the LockedInodePath releases

Returns:

a InodePathPair representing the two locked paths

Throws:

InvalidPathException - if a path is invalid

ensureFullInodePath

public void ensureFullInodePath(LockedInodePath inodePath)
                         throws InvalidPathException,
                                FileDoesNotExistException

Attempts to extend an existing LockedInodePath to reach the target inode (the last inode for the full path). If the target inode does not exist, an exception will be thrown.

Parameters:

inodePath - the LockedInodePath to extend to the target inode

Throws:

InvalidPathException - if the path is invalid

FileDoesNotExistException - if the target inode does not exist

getPathInodeNames

public ArrayList<String> getPathInodeNames(InodeView inode)
                                    throws FileDoesNotExistException

Computes the path components for the given inode view. This method should not be called while holding locks on any inodes. The operation is not guaranteed to be atomic.

Parameters:

inode - the inode view to calculate the path for

Returns:

the list of path components

Throws:

FileDoesNotExistException - if the path to the inode doesn't exist

getPathInodeNames

public ArrayList<String> getPathInodeNames(long id)
                                    throws FileDoesNotExistException

Computes the path components for the given inode id. This method should not be called while holding locks on any inodes. The operation is not guaranteed to be atomic.

Parameters:

id - the inode id to calculate the path for

Returns:

the list of path components

Throws:

FileDoesNotExistException - if the path to the inode doesn't exist

getPath

public AlluxioURI getPath(long id)
                   throws FileDoesNotExistException

Returns the path for a particular inode id. The inode and the path to the inode must already be locked.

Parameters:

id - the inode id to get the path for

Returns:

the AlluxioURI for the path of the inode

Throws:

FileDoesNotExistException - if the path does not exist

getPath

public AlluxioURI getPath(InodeView inode)
                   throws FileDoesNotExistException

Returns the path for a particular inode. The inode and the path to the inode must already be locked.

Parameters:

inode - the inode to get the path for

Returns:

the AlluxioURI for the path of the inode

Throws:

FileDoesNotExistException - if the path does not exist

getRoot

public InodeDirectory getRoot()

Returns:

the root inode

createPath

public List<Inode> createPath(RpcContext rpcContext,
                              LockedInodePath inodePath,
                              CreatePathContext<?,?> context)
                       throws FileAlreadyExistsException,
                              BlockInfoException,
                              InvalidPathException,
                              IOException,
                              FileDoesNotExistException

Creates a file or directory at path. This method expects that the last edge leading to the target inode to be write-locked. If the last existing inode in the path is /a/b/c and we want to create /a/b/c/d/e, the c->d edge must be write locked. On success, createPath attempts to push the write lock forward as far as possible. For the above example, createPath would take a write lock on d->e, and downgrade the c->d lock from a write lock to a read lock. This may not be possible if inodePath is a composite path which doesn't own the write lock. In that case no downgrade will occur.

Parameters:

rpcContext - the rpc context

inodePath - the path

context - method context

Returns:

a list of created inodes

Throws:

FileAlreadyExistsException - when there is already a file at path if we want to create a directory there

BlockInfoException - when blockSizeBytes is invalid

InvalidPathException - when path is invalid, for example, (1) when there is nonexistent necessary parent directories and recursive is false, (2) when one of the necessary parent directories is actually a file

FileDoesNotExistException - if the parent of the path does not exist and the recursive option is false

IOException

getDescendants

public LockedInodePathList getDescendants(LockedInodePath inodePath)

Returns LockedInodePaths for all descendants of inodePath.

Parameters:

inodePath - a locked inode path. It must be write-locked

Returns:

all descendants

deleteInode

public void deleteInode(RpcContext rpcContext,
                        LockedInodePath inodePath,
                        long opTimeMs)
                 throws FileDoesNotExistException

Deletes a single inode from the inode tree by removing it from the parent inode.

Parameters:

rpcContext - the rpc context

inodePath - the LockedInodePath to delete

opTimeMs - the operation time

Throws:

FileDoesNotExistException - if the Inode cannot be retrieved

setPinned

public void setPinned(RpcContext rpcContext,
                      LockedInodePath inodePath,
                      boolean pinned,
                      List<String> mediumTypes,
                      long opTimeMs)
               throws FileDoesNotExistException,
                      InvalidPathException

Sets the pinned state of an inode. If the inode is a directory, the pinned state will be set recursively.

Parameters:

rpcContext - the rpc context

inodePath - the LockedInodePath to set the pinned state for

pinned - the pinned state to set for the inode (and possible descendants)

mediumTypes - the list of pinned media that that the file can reside in

opTimeMs - the operation time

Throws:

FileDoesNotExistException - if inode does not exist

InvalidPathException

setReplication

public void setReplication(RpcContext rpcContext,
                           LockedInodePath inodePath,
                           Integer replicationMax,
                           Integer replicationMin,
                           long opTimeMs)
                    throws FileDoesNotExistException,
                           InvalidPathException

Sets the min and/or max replication level of an inode. If the inode is a directory, the state will be set recursively. Arguments replicationMax and replicationMin can be null if they are not meant to be set.

Parameters:

rpcContext - the rpc context

inodePath - the LockedInodePath to set the pinned state for

replicationMax - the max replication level to set for the inode (and possible descendants)

replicationMin - the min replication level to set for the inode (and possible descendants)

opTimeMs - the operation time

Throws:

FileDoesNotExistException - if inode does not exist

InvalidPathException

getReplicationLimitedFileIds

public Set<Long> getReplicationLimitedFileIds()

Returns:

the set of file ids whose replication max is not infinity

getToBePersistedIds

public Set<Long> getToBePersistedIds()

Returns:

an unmodifiable view of the files with persistence state PersistenceState.TO_BE_PERSISTED

getPinIdSet

public Set<Long> getPinIdSet()

Returns:

the set of file ids which are pinned

getInodeLockManager

public InodeLockManager getInodeLockManager()

Returns:

the inode lock manager for the inode tree

isRootId

public boolean isRootId(long fileId)

Parameters:

fileId - the file id to check

Returns:

true if the given file id is the root id

getDelegate

public Journaled getDelegate()

Specified by:

getDelegate in interface DelegatingJournaled

Returns:

the object to delegate to

syncPersistExistingDirectory

public void syncPersistExistingDirectory(java.util.function.Supplier<JournalContext> context,
                                         InodeDirectoryView dir,
                                         boolean isMetadataLoad)
                                  throws IOException,
                                         InvalidPathException,
                                         FileDoesNotExistException

Synchronously persists an inode directory to the UFS. If concurrent calls are made, only one thread will persist to UFS, and the others will wait until it is persisted.

Parameters:

context - journal context supplier

dir - the inode directory to persist

isMetadataLoad - if this operation is the result of a metadata load

Throws:

InvalidPathException - if the path for the inode is invalid

FileDoesNotExistException - if the path for the inode is invalid

IOException

syncPersistNewDirectory

public void syncPersistNewDirectory(MutableInodeDirectory dir,
                                    CreatePathContext<?,?> createContext)
                             throws InvalidPathException,
                                    FileDoesNotExistException,
                                    IOException

Synchronously persists an inode directory to the UFS. This method does not handle concurrent modification to the given inode, so the inode must not yet be added to the inode tree.

Parameters:

dir - the inode directory to persist

createContext - the create directory context

Throws:

InvalidPathException

FileDoesNotExistException

IOException

close

public void close()
           throws IOException

Close resources associated with this tree instance.

Throws:

IOException