Secure Alluxio has three features currently. This document describes the concepts and usage of them.
- Authentication: If enabled, Alluxio file system can recognize and verify the user accessing it. It is the basis for other security features such as authorization.
- Authorization: If enabled, Alluxio file system can control the user’s access. POSIX permission model is used in Alluxio to assign permissions and control access rights.
- Auditing: If enabled, Alluxio file system can maintain an audit log for users’ accesses to file metadata.
By default Alluxio runs in SIMPLE secure mode in which a simple authentication is required. SIMPLE indicates that server trusts whoever the client claims to be. See Security specific configuration to enable and use security features.
Alluxio provides file system service through Thrift RPC. The client side (representing a user) and the server side (such as master) should build an authenticated connection for communication. If authentication succeeds, the connection will be built. If fails, the connection can not be built and an exception will be thrown to client.
Three authentication modes are supported: NOSASL, SIMPLE (default mode), and CUSTOM.
The communication entities in Alluxio consist of master, worker, and client. Each of them needs to know its user who are running it, also called as the login user. JAAS (Java Authentication and Authorization Service) is used to determine who is currently executing the process.
When authentication is enabled, a login user for the component (master, worker, or client) can be obtained by following steps:
- Login by configurable user. If property ‘alluxio.security.login.username’ is set by application, its value will be the login user.
- If its value is empty, login by OS account.
If login fails, an exception will be thrown. If succeeds,
- For master, the login user is the super user of Alluxio file system. It is also the owner of root directory.
- For worker and client, the login user is the user who contacts with master for accessing file. It is passed to master through RPC connection for authentication.
Authentication is disabled. SASL (Simple Authentication and Security Layer) is a framework to define the authentication between client and server applications, which is used in Alluxio to implement authentication feature. So NOSASL is used to represent disabled case and Alluxio file system behavior is as before.
Authentication is enabled. Alluxio file system can know the user accessing it, and simply believe the user is the one he/she claims.
After a user creates directories/files, the user name is added into metadata. This user info could be read and shown in CLI and UI.
Authentication is enabled. Alluxio file system can know the user accessing it,
and use customized
AuthenticationProvider to verify the user is the one he/she claims.
Experimental. This mode is only used in tests currently.
Alluxio file system implements a permissions model for directories and files, which is similar as the POSIX permission model.
Each file and directory is associated with:
- an owner, which is the user of the client process to create the file or directory.
- a group, which is the group fetched from user-groups-mapping service. See User group mapping.
The permissions has three parts:
- owner permission defines the access privileges of the file owner
- group permission defines the access privileges of the owning group
- other permission defines the access privileges of all users that are not in any of above two classes
Each permission has three actions:
- read (r)
- write (w)
- execute (x)
For files, the r permission is required to read the file, and the w permission is required to write the file. For directories, the r permission is required to list the contents of the directory, the w permission is required to create, rename or delete files or directories under it, and the x permission is required to access a child of the directory.
For example, the output of the shell command
ls -R when authorization is enabled is:
./bin/alluxio fs ls -R / drwxr-xr-x jack staff 0.00B 02-02-2016 04:01:46:603 /default_tests_files -rw-r--r-- jack staff 80.00B 02-02-2016 04:01:46:603 In Memory /default_tests_files/BasicFile
User group mapping
When user is determined, the list of groups is determined by a group mapping service, configured by ‘alluxio.security.group.mapping.class’. The default implementation is ‘alluxio.security.group .provider.ShellBasedUnixGroupsMapping’, which executes the ‘groups’ shell command to fetch the group memberships of a given user. There is a caching mechanism for user group mapping, the mapping data will be cached for 60 seconds by default, this value can be configured by ‘alluxio.security.group.mapping.cache.timeout.ms’, if the value is ‘0’, the cached will be disabled.
Property ‘alluxio.security.authorization.permission.supergroup’ defines a super group. Any users belong to this group are also super users.
Initialized directory and file permissions
The initial creation permission is 777, and the difference between directory and file is 111. For default umask value 022, the created directory has permission 755 and file has permission 644. The umask can be set by property ‘alluxio.security.authorization.permission.umask’.
Update directory and file permission model
The owner, group, and permissions can be changed by two ways:
- User application invokes the setAttribute(…) method of
Hadoop API. See File System API.
- CLI command in shell. See chown, chgrp, chmod.
The owner can only be changed by super user. The group and permission can only be changed by super user and file owner.
Alluxio supports audit logging to allow system administrators to track users’ access to file metadata.
The audit log file (
master_audit.log) contains multiple audit log entries, each of which corresponds to an access to file metadata.
The format of Alluxio audit log entry is shown in the table below.
|succeeded||True if the command has succeeded. To succeed, it must also have been allowed.|
|allowed||True if the command has been allowed. Note that a command can still fail even if it has been allowed.|
|ugi||User group information, including username, primary group, and authentication type.|
|ip||Client IP address.|
|cmd||Command issued by the user.|
|src||Path of the source file or directory.|
|dst||Path of the destination file or directory. If not applicable, the value is null.|
|perm||User:group:mask or null if not applicable.|
It is similar to the format of HDFS audit log wiki.
To enable Alluxio audit logging, you need to set the JVM property
alluxio.master.audit.logging.enabled to true, see Configuration settings.
Service level encryption is not supported yet, user could encrypt sensitive data at application level, or enable encryption feature at under file system, e.g. HDFS transparent encryption, Linux disk encryption.
It is recommended to start Alluxio master and workers by one same user. Alluxio cluster service composes of master and workers. Every worker needs to RPC with master for some file operations. If the user of a worker is not the same as the master’s, the file operations may fail because of permission check.