@PublicApi @ThreadSafe public interface UnderFileSystem extends Closeable
Modifier and Type | Interface and Description |
---|---|
static class |
UnderFileSystem.Factory
The factory for the
UnderFileSystem . |
static class |
UnderFileSystem.SpaceType
The different types of space indicate the total space, the free space and the space used in the
under file system.
|
Modifier and Type | Method and Description |
---|---|
void |
cleanup()
Cleans up the under file system.
|
void |
connectFromMaster(String hostname)
Takes any necessary actions required to establish a connection to the under file system from
the given master host e.g.
|
void |
connectFromWorker(String hostname)
Takes any necessary actions required to establish a connection to the under file system from
the given worker host e.g.
|
OutputStream |
create(String path)
Creates a file in the under file system with the indicated name.
|
OutputStream |
create(String path,
CreateOptions options)
Creates a file in the under file system with the specified
CreateOptions . |
OutputStream |
createNonexistingFile(String path)
Creates a file in the under file system with the indicated name.
|
OutputStream |
createNonexistingFile(String path,
CreateOptions options)
Creates a file in the under file system with the specified
CreateOptions . |
boolean |
deleteDirectory(String path)
Deletes a directory from the under file system with the indicated name non-recursively.
|
boolean |
deleteDirectory(String path,
DeleteOptions options)
Deletes a directory from the under file system with the indicated name.
|
boolean |
deleteExistingDirectory(String path)
Deletes a directory from the under file system.
|
boolean |
deleteExistingDirectory(String path,
DeleteOptions options)
Deletes a directory from the under file system with the indicated name.
|
boolean |
deleteExistingFile(String path)
Deletes a file from the under file system with the indicated name.
|
boolean |
deleteFile(String path)
Deletes a file from the under file system with the indicated name.
|
boolean |
exists(String path)
Checks if a file or directory exists in under file system.
|
Pair<AccessControlList,DefaultAccessControlList> |
getAclPair(String path)
Gets the ACL and the Default ACL of a file or directory in under file system.
|
SyncInfo |
getActiveSyncInfo()
Return the active sync info for the specified syncPoints.
|
long |
getBlockSizeByte(String path)
Deprecated.
block size should be returned as part of the
UfsFileStatus from
getFileStatus(String) or getExistingFileStatus(String) |
default AlluxioConfiguration |
getConfiguration()
Gets the under file system configuration.
|
UfsDirectoryStatus |
getDirectoryStatus(String path)
Gets the directory status.
|
UfsDirectoryStatus |
getExistingDirectoryStatus(String path)
Gets the directory status.
|
UfsFileStatus |
getExistingFileStatus(String path)
Gets the file status.
|
UfsStatus |
getExistingStatus(String path)
Gets the file or directory status.
|
List<String> |
getFileLocations(String path)
Gets the list of locations of the indicated path.
|
List<String> |
getFileLocations(String path,
FileLocationOptions options)
Gets the list of locations of the indicated path given options.
|
UfsFileStatus |
getFileStatus(String path)
Gets the file status.
|
String |
getFingerprint(String path)
Deprecated.
|
UfsMode |
getOperationMode(Map<String,UfsMode> physicalUfsState)
An
UnderFileSystem may be composed of one or more "physical UFS"s. |
Fingerprint |
getParsedFingerprint(String path)
Computes and returns a fingerprint for the path.
|
Fingerprint |
getParsedFingerprint(String path,
String contentHash)
Same as
getParsedFingerprint(String) except, will use the given content hash
as the Fingerprint.Tag.CONTENT_HASH field of the fingerprint
if non-null. |
List<String> |
getPhysicalStores()
An
UnderFileSystem may be composed of one or more "physical UFS"s. |
long |
getSpace(String path,
UnderFileSystem.SpaceType type)
Queries the under file system about the space of the indicated path (e.g., space left, space
used and etc).
|
UfsStatus |
getStatus(String path)
Gets the file or directory status.
|
String |
getUnderFSType()
Returns the name of the under filesystem implementation.
|
boolean |
isDirectory(String path)
Checks if a directory exists in under file system.
|
boolean |
isExistingDirectory(String path)
Checks if a directory exists in under file system.
|
boolean |
isFile(String path)
Checks if a file exists in under file system.
|
boolean |
isObjectStorage() |
boolean |
isSeekable()
Denotes if the under storage supports seeking.
|
UfsStatus[] |
listStatus(String path)
Returns an array of statuses of the files and directories in the directory denoted by this
abstract pathname.
|
UfsStatus[] |
listStatus(String path,
ListOptions options)
Returns an array of statuses of the files and directories in the directory denoted by this
abstract pathname, with options.
|
boolean |
mkdirs(String path)
Creates the directory named by this abstract pathname.
|
boolean |
mkdirs(String path,
MkdirsOptions options)
Creates the directory named by this abstract pathname, with specified
MkdirsOptions . |
InputStream |
open(String path)
Opens an
InputStream for a file in under filesystem at the indicated path. |
InputStream |
open(String path,
OpenOptions options)
Opens an
InputStream for a file in under filesystem at the indicated path. |
InputStream |
openExistingFile(String path)
Opens an
InputStream for a file in under filesystem at the indicated path. |
InputStream |
openExistingFile(String path,
OpenOptions options)
Opens an
InputStream for a file in under filesystem at the indicated path. |
boolean |
renameDirectory(String src,
String dst)
Renames a directory from
src to dst in under file system. |
boolean |
renameFile(String src,
String dst)
Renames a file from
src to dst in under file system. |
boolean |
renameRenamableDirectory(String src,
String dst)
Renames a directory from
src to dst in under file system. |
boolean |
renameRenamableFile(String src,
String dst)
Renames a file from
src to dst in under file system. |
AlluxioURI |
resolveUri(AlluxioURI ufsBaseUri,
String alluxioPath)
Returns an
AlluxioURI representation for the UnderFileSystem given a base
UFS URI, and the Alluxio path from the base. |
void |
setAclEntries(String path,
List<AclEntry> aclEntries)
Sets the access control list of a file or directory in under file system.
|
void |
setMode(String path,
short mode)
Changes posix file mode.
|
void |
setOwner(String path,
String owner,
String group)
Sets the user and group of the given path.
|
boolean |
startActiveSyncPolling(long txId)
Start Active Sync.
|
void |
startSync(AlluxioURI uri)
Add Sync Point.
|
boolean |
stopActiveSyncPolling()
Stop Active Sync.
|
void |
stopSync(AlluxioURI uri)
Stop Sync Point.
|
boolean |
supportsActiveSync()
Whether this type of UFS supports active sync.
|
boolean |
supportsFlush()
Whether this type of UFS supports flush.
|
void cleanup() throws IOException
IOException
void connectFromMaster(String hostname) throws IOException
Depending on the implementation this may be a no-op
hostname
- the host that wants to connect to the under file systemIOException
void connectFromWorker(String hostname) throws IOException
Depending on the implementation this may be a no-op
hostname
- the host that wants to connect to the under file systemIOException
OutputStream create(String path) throws IOException
path
- the file nameOutputStream
objectIOException
OutputStream create(String path, CreateOptions options) throws IOException
CreateOptions
.
Implementations should make sure that the path under creation appears in listings only
after a successful close and that contents are written in its entirety or not at all.path
- the file nameoptions
- the options for createOutputStream
objectIOException
OutputStream createNonexistingFile(String path) throws IOException
create(String)
but
deals with the delete-then-create eventual consistency issue.path
- the file nameOutputStream
objectIOException
OutputStream createNonexistingFile(String path, CreateOptions options) throws IOException
CreateOptions
.
Similar to create(String, CreateOptions)
but
deals with the delete-then-create eventual consistency issue.path
- the file nameoptions
- the options for createOutputStream
objectIOException
boolean deleteDirectory(String path) throws IOException
path
- of the directory to deleteIOException
boolean deleteDirectory(String path, DeleteOptions options) throws IOException
path
- of the directory to deleteoptions
- for directory delete semanticsIOException
boolean deleteExistingDirectory(String path) throws IOException
deleteDirectory(String)
but
deals with the create-delete eventual consistency issue.path
- of the directory to deleteIOException
boolean deleteExistingDirectory(String path, DeleteOptions options) throws IOException
deleteDirectory(String, DeleteOptions)
but
deals with the create-then-delete eventual consistency issue.path
- of the directory to deleteoptions
- for directory delete semanticsIOException
boolean deleteFile(String path) throws IOException
path
- of the file to deleteIOException
boolean deleteExistingFile(String path) throws IOException
deleteFile(String)
but
deals with the create-then-delete eventual consistency issue.path
- of the file to deleteIOException
boolean exists(String path) throws IOException
path
- the absolute pathIOException
@Nullable Pair<AccessControlList,DefaultAccessControlList> getAclPair(String path) throws IOException
path
- the path to the file or directoryIOException
- if ACL is supported and enabled but cannot be retrieved@Deprecated long getBlockSizeByte(String path) throws IOException
UfsFileStatus
from
getFileStatus(String)
or getExistingFileStatus(String)
path
- the file nameIOException
default AlluxioConfiguration getConfiguration() throws IOException
IOException
UfsDirectoryStatus getDirectoryStatus(String path) throws IOException
path
- the path to the directoryFileNotFoundException
- when the path does not existIOException
UfsDirectoryStatus getExistingDirectoryStatus(String path) throws IOException
getDirectoryStatus(String)
but
deals with the write-then-get-status eventual consistency issue.path
- the path to the directoryIOException
List<String> getFileLocations(String path) throws IOException
path
- the file nameIOException
List<String> getFileLocations(String path, FileLocationOptions options) throws IOException
path
- the file nameoptions
- method optionsIOException
UfsFileStatus getFileStatus(String path) throws IOException
path
- the path to the fileFileNotFoundException
- when the path does not existIOException
UfsFileStatus getExistingFileStatus(String path) throws IOException
getFileStatus(String)
but
deals with the write-then-get-status eventual consistency issue.path
- the path to the fileIOException
@Deprecated String getFingerprint(String path)
Constants.INVALID_UFS_FINGERPRINT
if there is any error.path
- the path to compute the fingerprint forFingerprint getParsedFingerprint(String path)
Fingerprint.INVALID_FINGERPRINT
if there is any error.
The default implementation relies on getFingerprint(String)
and there is one extra
parsing. This default implementation is mainly for backward compatibility.
Override this for performance.path
- the path to compute the fingerprint forFingerprint getParsedFingerprint(String path, @Nullable String contentHash)
getParsedFingerprint(String)
except, will use the given content hash
as the Fingerprint.Tag.CONTENT_HASH
field of the fingerprint
if non-null. This is intended to be used when the file is already in Alluxio and
a fingerprint is being created based on that file where the content hash has already
been computed.path
- the path to compute the fingerprint forcontentHash
- is used as the Fingerprint.Tag.CONTENT_HASH
field when creating the fingerprint.UfsMode getOperationMode(Map<String,UfsMode> physicalUfsState)
UnderFileSystem
may be composed of one or more "physical UFS"s. This method is used
to determine the operation mode based on the physical UFS operation modes. For example, if this
UnderFileSystem
is composed of physical UFS hdfs://ns1/ and hdfs://ns2/ with read
operations split b/w the two, with physicalUfsState{hdfs://ns1/:NO_ACCESS,
hdfs://ns2/:READ_WRITE} this method can return READ_ONLY to allow reads to proceed from
hdfs://ns2/.physicalUfsState
- the state of physical UFSs for this UnderFileSystem
; keys are
expected to be normalized (ending with /)List<String> getPhysicalStores()
UnderFileSystem
may be composed of one or more "physical UFS"s. This method
returns all underlying physical stores; normalized with only scheme and authority.UnderFileSystem
is composed oflong getSpace(String path, UnderFileSystem.SpaceType type) throws IOException
path
- the path to querytype
- the type of queriesIOException
UfsStatus getStatus(String path) throws IOException
path
- the path to get the statusFileNotFoundException
- when the path does not existIOException
UfsStatus getExistingStatus(String path) throws IOException
getStatus(String)
but
deals with the write-then-get-status eventual consistency issue.path
- the path to get the statusIOException
String getUnderFSType()
boolean isDirectory(String path) throws IOException
path
- the absolute directory pathIOException
boolean isExistingDirectory(String path) throws IOException
isDirectory(String)
but
deals with the write-then-list eventual consistency issue.path
- the absolute directory pathIOException
boolean isFile(String path) throws IOException
path
- the absolute file pathIOException
boolean isObjectStorage()
boolean isSeekable()
SeekableUnderFileInputStream
in the open(String, OpenOptions)
method.@Nullable UfsStatus[] listStatus(String path) throws IOException
If this abstract pathname does not denote a directory, then this method returns null
.
Otherwise an array of statuses is returned, one for each file or directory in the directory.
Names denoting the directory itself and the directory's parent directory are not included in
the result. Each string is a file name rather than a complete path.
There is no guarantee that the name strings in the resulting array will appear in any specific order; they are not, in particular, guaranteed to appear in alphabetical order.
path
- the abstract pathname to listnull
if this abstract pathname does not denote a directory.IOException
@Nullable UfsStatus[] listStatus(String path, ListOptions options) throws IOException
If this abstract pathname does not denote a directory, then this method returns null
.
Otherwise an array of statuses is returned, one for each file or directory. Names denoting the
directory itself and the directory's parent directory are not included in the result. Each
string is a path relative to the given directory.
There is no guarantee that the name strings in the resulting array will appear in any specific order; they are not, in particular, guaranteed to appear in alphabetical order.
path
- the abstract pathname to listoptions
- for list directorynull
if this abstract pathname does not denote a directory.IOException
boolean mkdirs(String path) throws IOException
path
- the folder to createtrue
if and only if the directory was created; false
otherwiseIOException
boolean mkdirs(String path, MkdirsOptions options) throws IOException
MkdirsOptions
. If the folder already exists, the method returns false.path
- the folder to createoptions
- the options for mkdirstrue
if and only if the directory was created; false
otherwiseIOException
InputStream open(String path) throws IOException
InputStream
for a file in under filesystem at the indicated path.path
- the file nameInputStream
objectIOException
InputStream open(String path, OpenOptions options) throws IOException
InputStream
for a file in under filesystem at the indicated path.path
- the file nameoptions
- to open input streamInputStream
objectIOException
InputStream openExistingFile(String path) throws IOException
InputStream
for a file in under filesystem at the indicated path.
Similar to open(String)
but
deals with the write-then-read eventual consistency issue.path
- the file nameInputStream
objectIOException
InputStream openExistingFile(String path, OpenOptions options) throws IOException
InputStream
for a file in under filesystem at the indicated path.
Similar to open(String, OpenOptions)
but
deals with the write-then-read eventual consistency issue.path
- the file nameoptions
- to open input streamInputStream
objectIOException
boolean renameDirectory(String src, String dst) throws IOException
src
to dst
in under file system.src
- the source directory pathdst
- the destination directory pathIOException
boolean renameRenamableDirectory(String src, String dst) throws IOException
src
to dst
in under file system.
Similar to renameDirectory(String, String)
but
deals with the write-src-then-rename and delete-dst-then-rename eventual consistency issue.src
- the source directory pathdst
- the destination directory pathIOException
boolean renameFile(String src, String dst) throws IOException
src
to dst
in under file system.src
- the source file pathdst
- the destination file pathIOException
boolean renameRenamableFile(String src, String dst) throws IOException
src
to dst
in under file system.
Similar to renameFile(String, String)
but
deals with the write-src-then-rename and delete-dst-then-rename eventual consistency issue.src
- the source file pathdst
- the destination file pathIOException
AlluxioURI resolveUri(AlluxioURI ufsBaseUri, String alluxioPath)
AlluxioURI
representation for the UnderFileSystem
given a base
UFS URI, and the Alluxio path from the base.
The default implementation simply concatenates the path to the base URI. This should be
overridden if a subclass needs alternate functionality.ufsBaseUri
- the base AlluxioURI
in the ufsalluxioPath
- the path in Alluxio from the given baseAlluxioURI
representing the Alluxio pathvoid setAclEntries(String path, List<AclEntry> aclEntries) throws IOException
path
- the path to the file or directoryaclEntries
- the access control list + default acl represented in a list of acl entriesIOException
void setMode(String path, short mode) throws IOException
path
- the path of the filemode
- the mode to set in short format, e.g. 0777IOException
void setOwner(String path, String owner, String group) throws IOException
path
- the path of the fileowner
- the new owner to set, unchanged if nullgroup
- the new group to set, unchanged if nullIOException
boolean supportsFlush() throws IOException
IOException
boolean supportsActiveSync()
SyncInfo getActiveSyncInfo() throws IOException
IOException
void startSync(AlluxioURI uri) throws IOException
uri
- ufs uri to startIOException
void stopSync(AlluxioURI uri) throws IOException
uri
- ufs uri to stopIOException
boolean startActiveSyncPolling(long txId) throws IOException
txId
- the transaction id to start receiving eventIOException
boolean stopActiveSyncPolling() throws IOException
IOException
Copyright © 2023. All Rights Reserved.