Developer Tips
- Change a Thrift RPC definition
- Change a Protocol Buffer Message
- Full list of the commands in bin/alluxio
This page is a collection of tips and howtos geared towards developers of Alluxio.
Change a Thrift RPC definition
Alluxio uses thrift for RPC communication between clients and servers. The .thrift
files defined in common/src/thrift/
are used to auto-generate Java code for calling the
RPCs on clients and implementing the RPCs on servers. To change a Thrift definition, you
must first install the Thrift compiler.
If you have brew, you can do this by running
brew install thrift
Then to regenerate the Java code, run
bin/alluxio thriftGen
Change a Protocol Buffer Message
Alluxio uses protocol buffers to read and write journal messages. The .proto
files
defined in servers/src/proto/journal/
are used to auto-generate Java definitions for
the protocol buffer messages. To change one of these messages, first read about
updating a message type
to make sure your change will not break backwards compatibility. Next,
install protoc.
If you have brew, you can do this by running
brew install protobuf
Then to regenerate the Java code, run
bin/alluxio protoGen
Full list of the commands in bin/alluxio
Most commands in bin/alluxio
are for developers. The following table explains the description and
the syntax of each command.
Command | Args | Description |
---|---|---|
format | [-s] | Format Alluxio Master and all Workers. The option [-s] indicates that the command should only format when the underfs is local and doesn't already exist. |
formatWorker | None | Format Alluxio Worker storage on this local node. |
bootstrapConf | <ALLUXIO_MASTER_HOSTNAME> | Generate the bootstrap config file alluxio-env.sh with the specified ALLUXIO_MASTER_HOSTNAME , if the config file doesn't exist. |
fs | [fs-commands] | Interact with Alluxio in command line style for basic file system operations. See Command Line for more information. |
loadufs | <TachyonPath> <UfsPath> [ExcludePathPrefixes] | Loads files under UfsPath to the given AlluxioPath . ExcludePathPrefixes can be a set of prefixes which are separated by ';'. The paths with a prefix in ExcludePathPrefixes will not be loaded. |
runTest | <Example> <ReadType> <WriteType> | Run an end-to-end test on a Alluxio cluster. Example should be "Basic" or "BasicNonByteBuffer". ReadType should be "CACHE_PROMOTE", "CACHE", or "NO_CACHE". WriteType should be "MUST_CACHE", "CACHE_THROUGH" or "THROUGH". |
runTests | None | Run all end-to-end tests on a Alluxio cluster. That is, execute the runTest command with all the possible args. |
journalCrashTest | [-creates <arg>] [-deletes <arg>] [-renames <arg>] [-maxAlive <arg>] [-testDir <arg>] [-totalTime <arg>] [-help] | Test the Master Journal System in a crash scenario. Try alluxio journalCrashTest -help to see the meanings of each argument in detail, or you can run it without args by default. |
readJournal | [-help] [-noTimeout] | Read a Alluxio journal file from stdin and write a human-readable version of it to stdout. You can run this on the journal file as alluxio readJournal < journal/FileSystemMaster/log.out . |
killAll | <WORD> | Kill processes whose pid or command contains the WORD specified by the user. |
copyDir | <PATH> | Copy the PATH to all worker nodes. |
clearCache | None | Clear OS buffer cache of the machine. This command needs the root permission. |
thriftGen | None | Generate all thrift code. See Change a Thrift RPC definition. |
protoGen | None | Generate all protocol buffer code. See Change a Protocol Buffer Message. |
version | None | Print Alluxio version. |
validateConf | None | Validate Alluxio conf. |
In addition, these commands have different prerequisites. The prerequisite for the format
,
formatWorker
, journalCrashTest
, readJournal
, version
and validateConf
commands is that you
have already built Alluxio (see Build Alluxio Master Branch
about how to build Alluxio manually). Further, the prerequisite for the fs
, loadufs
, runTest
and runTests
commands is that you have a running Alluxio system.