S3 API

使用示例

S3 API 操作

AWS CLI
REST Clients

您可以使用 AWS 命令行接口向 Alluxio S3 API 发送 S3 API 请求。请注意，您必须提供 --endpoint参数来指定 Alluxio S3 REST API 的位置，并包含服务器的基本 URI。端点格式为 --endpoint "http://{alluxio.worker.web.hostname}:{alluxio.worker.rest.port}/"。

如涉及 授权 标头操作，您可能需要先配置 AWS 凭证。

有关 Alluxio 如何使用该标头的详细信息，请参阅授权标头

$ aws configure --profile alluxio-s3
AWS Access Key ID [None]: {user}
AWS Secret Access Key [None]: {dummy value}
Default region name [None]:
Default output format [None]:

您可以直接使用任何 HTTP client 向 Alluxio S3 API 发送 S3 API 请求。基本 URI 为 /；请求应指向 http://{alluxio.worker.web.hostname}:{alluxio.worker.rest.port}/。

目前，Alluxio S3 API 不存在access key和secret key验证。因此，[授权标头]（../api/S3-API.html#global-request-headers）仅用于指定执行请求的目标用户。 The header follows the AWS Signature Version 4 format.

$ curl -i -H "Authorization: AWS4-HMAC-SHA256 Credential=testuser/... SignedHeaders=... Signature=..." ...

$ aws --profile alluxio-s3 --endpoint "http://localhost:29998/" s3api copy-object \
  --copy-source=testbucket/test.txt --bucket=testbucket --key=test_copy.txt
{
    "CopyObjectResult": {
        "ETag": "911df44b7ff57801ca8d74568e4ebfbe",
        "LastModified": "2022-05-03T11:37:16.015000+00:00"
    }
}

$ aws --profile alluxio-s3 --endpoint "http://localhost:29998/" s3api list-objects \
  --bucket=testbucket
{
    "Contents": [
        {
            "Key": "test.txt",
            "LastModified": "2022-05-03T11:35:59.243000+00:00",
            "Size": 27040
        },
        {
            "Key": "test_copy.txt",
            "LastModified": "2022-05-03T11:37:16.185000+00:00",
            "Size": 27040
        }
    ]
}

$ curl -i -H "Authorization: AWS4-HMAC-SHA256 Credential=testuser/... SignedHeaders=... Signature=..." \
  -H "x-amz-copy-source: testbucket/test.txt" \
  -X PUT http://localhost:29998/testbucket/test_copy.txt
HTTP/1.1 200 OK
Date: Tue, 03 May 2022 21:50:07 GMT
Content-Type: application/xml
Content-Length: 135
Server: Jetty(9.4.43.v20210629)

<CopyObjectResult>
  <ETag>911df44b7ff57801ca8d74568e4ebfbe</ETag>
  <LastModified>2022-05-03T14:50:07.781Z</LastModified>
</CopyObjectResult>

$ curl -i -H "Authorization: AWS4-HMAC-SHA256 Credential=testuser/... SignedHeaders=... Signature=..." \
  -X GET http://localhost:29998/testbucket
HTTP/1.1 200 OK
Date: Tue, 03 May 2022 21:50:26 GMT
Content-Type: application/xml
Content-Length: 434
Server: Jetty(9.4.43.v20210629)

<ListBucketResult>
  <version2>false</version2>
  <Contents>
    <Key>test.txt</Key>
    <Size>27040</Size>
    <LastModified>2022-05-03T14:47:36.600Z</LastModified>
  </Contents>
  <Contents>
    <Key>test_copy.txt</Key>
    <Size>27040</Size>
    <LastModified>2022-05-03T14:50:07.790Z</LastModified>
  </Contents>
  <Marker/>
  <IsTruncated>false</IsTruncated>
  <Prefix/>
  <Name>testbucket</Name>
  <MaxKeys>1000</MaxKeys>
  <EncodingType>url</EncodingType>
</ListBucketResult>

CreateBucket

AWS CLI
REST Clients

$ aws --profile alluxio-s3 --endpoint "http://localhost:29998/" s3api create-bucket \
  --bucket=testbucket

$ aws --profile alluxio-s3 --endpoint "http://localhost:29998/" s3api list-buckets
{
    "Buckets": [
        {
            "Name": "testbucket",
            "CreationDate": "2022-05-03T11:32:34.156000+00:00"
        }
    ]
}

$ curl -i -H "Authorization: AWS4-HMAC-SHA256 Credential=testuser/... SignedHeaders=... Signature=..." \
  -X PUT http://localhost:29998/testbucket

HTTP/1.1 200 OK
Date: Tue, 03 May 2022 21:35:05 GMT
Content-Length: 0
Server: Jetty(9.4.43.v20210629)

$ curl -i -H "Authorization: AWS4-HMAC-SHA256 Credential=testuser/... SignedHeaders=... Signature=..." \
  -X GET http://localhost:29998/
HTTP/1.1 200 OK
Date: Tue, 03 May 2022 21:35:23 GMT
Content-Type: application/xml
Content-Length: 161
Server: Jetty(9.4.43.v20210629)

<ListAllMyBucketsResult>
  <Buckets>
    <Bucket>
      <Name>testbucket</Name>
      <CreationDate>2022-05-03T14:34:56.420Z</CreationDate>
    </Bucket>
  </Buckets>
</ListAllMyBucketsResult>

DeleteBucket

AWS CLI
REST Clients

$ aws --profile alluxio-s3 --endpoint "http://localhost:29998/" s3api list-buckets
{
    "Buckets": [
        {
            "Name": "tempbucket",
            "CreationDate": "2022-05-03T11:55:58.134000+00:00"
        },
        {
            "Name": "testbucket",
            "CreationDate": "2022-05-03T11:32:34.156000+00:00"
        }
    ]
}

$ aws --profile alluxio-s3 --endpoint "http://localhost:29998/" s3api delete-bucket \
  --bucket=tempbucket

$ aws --profile alluxio-s3 --endpoint "http://localhost:29998/" s3api list-buckets
{
    "Buckets": [
        {
            "Name": "testbucket",
            "CreationDate": "2022-05-03T11:32:34.156000+00:00"
        }
    ]
}

$ curl -i -H "Authorization: AWS4-HMAC-SHA256 Credential=testuser/... SignedHeaders=... Signature=..." \
  -X GET http://localhost:29998/
HTTP/1.1 200 OK
Date: Tue, 03 May 2022 21:43:20 GMT
Content-Type: application/xml
Content-Length: 254
Server: Jetty(9.4.43.v20210629)

<ListAllMyBucketsResult>
  <Buckets>
    <Bucket>
      <Name>tempbucket</Name>
      <CreationDate>2022-05-03T14:43:03.651Z</CreationDate>
    </Bucket>
    <Bucket>
      <Name>testbucket</Name>
      <CreationDate>2022-05-03T14:34:56.420Z</CreationDate>
    </Bucket>
  </Buckets>
</ListAllMyBucketsResult>

$ curl -i -H "Authorization: AWS4-HMAC-SHA256 Credential=testuser/... SignedHeaders=... Signature=..." \
  -X DELETE http://localhost:29998/tempbucket
HTTP/1.1 204 No Content
Date: Tue, 03 May 2022 21:43:25 GMT
Server: Jetty(9.4.43.v20210629)

$ curl -i -H "Authorization: AWS4-HMAC-SHA256 Credential=testuser/... SignedHeaders=... Signature=..." \
  -X GET http://localhost:29998/
HTTP/1.1 200 OK
Date: Tue, 03 May 2022 21:43:28 GMT
Content-Type: application/xml
Content-Length: 161
Server: Jetty(9.4.43.v20210629)

<ListAllMyBucketsResult>
  <Buckets>
    <Bucket>
      <Name>testbucket</Name>
      <CreationDate>2022-05-03T14:34:56.420Z</CreationDate>
    </Bucket>
  </Buckets>
</ListAllMyBucketsResult>

DeleteBucketTagging

AWS CLI
REST Clients

$ aws --profile alluxio-s3 --endpoint "http://localhost:29998/" s3api get-bucket-tagging \
  --bucket=testbucket
{
    "TagSet": [
        {
            "Key": "key1",
            "Value": "val1"
        },
        {
            "Key": "key2",
            "Value": "val2"
        }
    ]
}

$ aws --profile alluxio-s3 --endpoint "http://localhost:29998/" s3api delete-bucket-tagging \
  --bucket=testbucket

$ aws --profile alluxio-s3 --endpoint "http://localhost:29998/" s3api get-bucket-tagging \
  --bucket=testbucket
{
    "TagSet": []
}

$ curl -i -H "Authorization: AWS4-HMAC-SHA256 Credential=testuser/... SignedHeaders=... Signature=..." \
  -X GET "http://localhost:29998/testbucket?tagging"
HTTP/1.1 200 OK
Date: Tue, 03 May 2022 23:31:07 GMT
Content-Type: application/xml
Content-Length: 124
Server: Jetty(9.4.43.v20210629)

<Tagging>
  <TagSet>
    <Tag>
      <Key>key1</Key>
      <Value>val1</Value>
    </Tag>
    <Tag>
      <Key>key2</Key>
      <Value>val2</Value>
    </Tag>
  </TagSet>
</Tagging>

$ curl -i -H "Authorization: AWS4-HMAC-SHA256 Credential=testuser/... SignedHeaders=... Signature=..." \
  -X DELETE "http://localhost:29998/testbucket?tagging"
HTTP/1.1 204 No Content
Date: Tue, 03 May 2022 23:32:26 GMT
Server: Jetty(9.4.43.v20210629)

$ curl -i -H "Authorization: AWS4-HMAC-SHA256 Credential=testuser/... SignedHeaders=... Signature=..." \
  -X GET "http://localhost:29998/testbucket?tagging"
HTTP/1.1 200 OK
Date: Tue, 03 May 2022 23:32:27 GMT
Content-Type: application/xml
Content-Length: 28
Server: Jetty(9.4.43.v20210629)

<Tagging><TagSet/></Tagging>

DeleteObject

AWS CLI
REST Clients

$ aws --profile alluxio-s3 --endpoint "http://localhost:29998/" s3api list-objects-v2 \
  --bucket=testbucket
{
    "Contents": [
        {
            "Key": "temp.txt",
            "LastModified": "2022-05-03T11:55:01.925000+00:00",
            "Size": 27040
        },
        {
            "Key": "test.txt",
            "LastModified": "2022-05-03T11:54:19.698000+00:00",
            "Size": 27040
        }
    ]
}

$ aws --profile alluxio-s3 --endpoint "http://localhost:29998/" s3api delete-object \
  --bucket=testbucket --key=temp.txt

$ aws --profile alluxio-s3 --endpoint "http://localhost:29998/" s3api list-objects-v2 \
  --bucket=testbucket
{
    "Contents": [
        {
            "Key": "test.txt",
            "LastModified": "2022-05-03T11:55:01.925000+00:00",
            "Size": 27040
        }
    ]
}

$ curl -i -H "Authorization: AWS4-HMAC-SHA256 Credential=testuser/... SignedHeaders=... Signature=..." \
  -X GET http://localhost:29998/testbucket
HTTP/1.1 200 OK
Date: Tue, 03 May 2022 21:59:27 GMT
Content-Type: application/xml
Content-Length: 540
Server: Jetty(9.4.43.v20210629)

<ListBucketResult>
  <version2>false</version2>
  <Contents>
    <Key>temp.txt</Key>
    <Size>27040</Size>
    <LastModified>2022-05-03T14:50:07.790Z</LastModified>
  </Contents>
  <Contents>
    <Key>test.txt</Key>
    <Size>27040</Size>
    <LastModified>2022-05-03T14:47:36.600Z</LastModified>
  </Contents>
  <Marker/>
  <IsTruncated>false</IsTruncated>
  <Prefix/>
  <Name>testbucket</Name>
  <MaxKeys>1000</MaxKeys>
  <EncodingType>url</EncodingType>
</ListBucketResult>

$ curl -i -H "Authorization: AWS4-HMAC-SHA256 Credential=testuser/... SignedHeaders=... Signature=..." \
  -X DELETE http://localhost:29998/testbucket/temp.txt
HTTP/1.1 204 No Content
Date: Tue, 03 May 2022 22:01:56 GMT
Server: Jetty(9.4.43.v20210629)

$ curl -i -H "Authorization: AWS4-HMAC-SHA256 Credential=testuser/... SignedHeaders=... Signature=..." \
  -X GET http://localhost:29998/testbucket
HTTP/1.1 200 OK
Date: Tue, 03 May 2022 22:01:59 GMT
Content-Type: application/xml
Content-Length: 318
Server: Jetty(9.4.43.v20210629)

<ListBucketResult>
  <version2>false</version2>
  <Contents>
    <Key>test.txt</Key>
    <Size>27040</Size>
    <LastModified>2022-05-03T14:47:36.600Z</LastModified>
  </Contents>
  <Marker/>
  <IsTruncated>false</IsTruncated>
  <Prefix/>
  <Name>testbucket</Name>
  <MaxKeys>1000</MaxKeys>
  <EncodingType>url</EncodingType>
</ListBucketResult>

DeleteObjects

AWS CLI
REST Clients

$ aws --profile alluxio-s3 --endpoint "http://localhost:29998/" s3api list-objects-v2 \
  --bucket=tempbucket
{
    "Contents": [
        {
            "Key": "foo.txt",
            "LastModified": "2022-05-03T11:57:00.767000+00:00",
            "Size": 27040
        },
        {
            "Key": "temp.txt",
            "LastModified": "2022-05-03T11:56:11.245000+00:00",
            "Size": 27040
        },
        {
            "Key": "temp2.txt",
            "LastModified": "2022-05-03T11:56:31.414000+00:00",
            "Size": 27040
        }
    ]
}

$ aws --profile alluxio-s3 --endpoint "http://localhost:29998/" s3api delete-objects \
  --bucket=tempbucket --delete="Objects=[{Key=temp.txt},{Key=temp2.txt}]"

$ aws --profile alluxio-s3 --endpoint "http://localhost:29998/" s3api list-objects-v2 \
  --bucket=tempbucket
{
    "Contents": [
        {
            "Key": "foo.txt",
            "LastModified": "2022-05-03T11:57:00.767000+00:00",
            "Size": 27040
        }
    ]
}

$ curl -i -H "Authorization: AWS4-HMAC-SHA256 Credential=testuser/... SignedHeaders=... Signature=..." \
  -X GET http://localhost:29998/tempbucket
HTTP/1.1 200 OK
Date: Tue, 03 May 2022 21:59:27 GMT
Content-Type: application/xml
Content-Length: 540
Server: Jetty(9.4.43.v20210629)

<ListBucketResult>
  <version2>false</version2>
  <Contents>
    <Key>foo.txt</Key>
    <Size>27040</Size>
    <LastModified>2022-05-03T14:59:05.906Z</LastModified>
  </Contents>
  <Contents>
    <Key>temp.txt</Key>
    <Size>27040</Size>
    <LastModified>2022-05-03T14:58:58.204Z</LastModified>
  </Contents>
  <Contents>
    <Key>temp2.txt</Key>
    <Size>27040</Size>
    <LastModified>2022-05-03T14:59:01.987Z</LastModified>
  </Contents>
  <Marker/>
  <IsTruncated>false</IsTruncated>
  <Prefix/>
  <Name>tempbucket</Name>
  <MaxKeys>1000</MaxKeys>
  <EncodingType>url</EncodingType>
</ListBucketResult>

$ cat delete.xml
<Delete xmlns="http://s3.amazonaws.com/doc/2006-03-01/">
  <Object>
    <Key>temp.txt</Key>
  </Object>
  <Object>
    <Key>temp2.txt</Key>
  </Object>
  <Quiet>false</Quiet>
</Delete>

$ curl -i -H "Authorization: AWS4-HMAC-SHA256 Credential=testuser/... SignedHeaders=... Signature=..." \
  -H "Content-Type: application/xml" \
  -X POST --data "@delete.xml" "http://localhost:29998/testbucket?delete"
HTTP/1.1 200 OK
Date: Tue, 03 May 2022 22:56:17 GMT
Content-Type: application/xml
Content-Length: 208
Server: Jetty(9.4.43.v20210629)

<DeleteResult>
  <Deleted>
    <Key>temp2.txt</Key>
    <DeleteMarker/>
    <DeleteMarkerVersionId/>
    <VersionId/>
  </Deleted>
  <Deleted>
    <Key>temp.txt</Key>
    <DeleteMarker/>
    <DeleteMarkerVersionId/>
    <VersionId/>
  </Deleted>
</DeleteResult>

$ curl -i -H "Authorization: AWS4-HMAC-SHA256 Credential=testuser/... SignedHeaders=... Signature=..." \
  -X GET http://localhost:29998/tempbucket
HTTP/1.1 200 OK
Date: Tue, 03 May 2022 22:28:31 GMT
Content-Type: application/xml
Content-Length: 317
Server: Jetty(9.4.43.v20210629)

<ListBucketResult>
  <version2>false</version2>
  <Contents>
    <Key>foo.txt</Key>
    <Size>27040</Size>
    <LastModified>2022-05-03T14:59:05.906Z</LastModified>
  </Contents>
  <Marker/>
  <IsTruncated>false</IsTruncated>
  <Prefix/>
  <Name>tempbucket</Name>
  <MaxKeys>1000</MaxKeys>
  <EncodingType>url</EncodingType>
</ListBucketResult>

DeleteObjectTagging

AWS CLI
REST Clients

$ aws --profile alluxio-s3 --endpoint "http://localhost:29998/" s3api get-object-tagging \
  --bucket=testbucket --key=test.txt
{
    "TagSet": [
        {
            "Key": "key1",
            "Value": "val1"
        },
        {
            "Key": "key2",
            "Value": "val2"
        }
    ]
}

$ aws --profile alluxio-s3 --endpoint "http://localhost:29998/" s3api delete-object-tagging \
  --bucket=testbucket --key=test.txt

$ aws --profile alluxio-s3 --endpoint "http://localhost:29998/" s3api get-object-tagging \
  --bucket=testbucket --key=test.txt
{
    "TagSet": []
}

$ curl -i -H "Authorization: AWS4-HMAC-SHA256 Credential=testuser/... SignedHeaders=... Signature=..." \
  -X GET "http://localhost:29998/testbucket?tagging"
HTTP/1.1 200 OK
Date: Tue, 03 May 2022 23:31:07 GMT
Content-Type: application/xml
Content-Length: 124
Server: Jetty(9.4.43.v20210629)

<Tagging>
  <TagSet>
    <Tag>
      <Key>key1</Key>
      <Value>val1</Value>
    </Tag>
    <Tag>
      <Key>key2</Key>
      <Value>val2</Value>
    </Tag>
  </TagSet>
</Tagging>

$ curl -i -H "Authorization: AWS4-HMAC-SHA256 Credential=testuser/... SignedHeaders=... Signature=..." \
  -X DELETE "http://localhost:29998/testbucket/test.txt?tagging"
HTTP/1.1 204 No Content
Date: Tue, 03 May 2022 23:37:46 GMT
Server: Jetty(9.4.43.v20210629)

$ curl -i -H "Authorization: AWS4-HMAC-SHA256 Credential=testuser/... SignedHeaders=... Signature=..." \
  -X GET "http://localhost:29998/testbucket/test.txt?tagging"
HTTP/1.1 200 OK
Date: Tue, 03 May 2022 23:37:47 GMT
Content-Type: application/octet-stream
Content-Length: 28
Server: Jetty(9.4.43.v20210629)

<Tagging><TagSet/></Tagging>

GetBucketTagging

AWS CLI
REST Clients

$ aws --profile alluxio-s3 --endpoint "http://localhost:29998/" s3api get-bucket-tagging \
  --bucket=testbucket
{
    "TagSet": [
        {
            "Key": "key1",
            "Value": "val1"
        },
        {
            "Key": "key2",
            "Value": "val2"
        }
    ]
}

$ curl -i -H "Authorization: AWS4-HMAC-SHA256 Credential=testuser/... SignedHeaders=... Signature=..." \
  -X GET "http://localhost:29998/testbucket?tagging"
HTTP/1.1 200 OK
Date: Tue, 03 May 2022 23:31:07 GMT
Content-Type: application/xml
Content-Length: 124
Server: Jetty(9.4.43.v20210629)

<Tagging>
  <TagSet>
    <Tag>
      <Key>key1</Key>
      <Value>val1</Value>
    </Tag>
    <Tag>
      <Key>key2</Key>
      <Value>val2</Value>
    </Tag>
  </TagSet>
</Tagging>

GetObject

AWS CLI
REST Clients

$ aws --profile alluxio-s3 --endpoint "http://localhost:29998/" s3api get-object \
  --bucket=testbucket --key=test.txt /tmp/test.txt
{
    "LastModified": "2022-05-03T18:55:01+00:00",
    "ContentLength": 27040,
    "ETag": "0cc175b9c0f1b6a831c399e269772661",
    "ContentType": "application/octet-stream",
    "Metadata": {}
}

$ stat /tmp/test.txt
  File: /tmp/test.txt
  Size: 27040       Blocks: 56         IO Block: 4096   regular file
  ...

$ curl -i -H "Authorization: AWS4-HMAC-SHA256 Credential=testuser/... SignedHeaders=... Signature=..." \
  -X GET http://localhost:29998/testbucket/test.txt
HTTP/1.1 200 OK
Date: Tue, 03 May 2022 22:59:43 GMT
Last-Modified: Tue, 03 May 2022 21:47:36 GMT
ETag: 0cc175b9c0f1b6a831c399e269772661
Content-Type: application/octet-stream
Content-Length: 27040
Server: Jetty(9.4.43.v20210629)

................. file contents .................

GetObjectTagging

AWS CLI
REST Clients

$ aws --profile alluxio-s3 --endpoint "http://localhost:29998/" s3api get-object-tagging \
  --bucket=testbucket --key=test.txt
{
    "TagSet": [
        {
            "Key": "key1",
            "Value": "val1"
        },
        {
            "Key": "key2",
            "Value": "val2"
        }
    ]
}

$ curl -i -H "Authorization: AWS4-HMAC-SHA256 Credential=testuser/... SignedHeaders=... Signature=..." \
  -X GET "http://localhost:29998/testbucket?tagging"
HTTP/1.1 200 OK
Date: Tue, 03 May 2022 23:31:07 GMT
Content-Type: application/xml
Content-Length: 124
Server: Jetty(9.4.43.v20210629)

<Tagging>
  <TagSet>
    <Tag>
      <Key>key1</Key>
      <Value>val1</Value>
    </Tag>
    <Tag>
      <Key>key2</Key>
      <Value>val2</Value>
    </Tag>
  </TagSet>
</Tagging>

HeadBucket

AWS CLI
REST Clients

$ aws --profile alluxio-s3 --endpoint "http://localhost:29998/" s3api head-bucket \
  --bucket=testbucket

$ curl -i -H "Authorization: AWS4-HMAC-SHA256 Credential=testuser/... SignedHeaders=... Signature=..." \
  --head http://localhost:29998/testbucket
HTTP/1.1 200 OK
Date: Tue, 15 Nov 2022 04:49:12 GMT
Content-Type: application/xml
Content-Length: 0
Server: Jetty(9.4.43.v20210629)

HeadObject

AWS CLI
REST Clients

$ aws --profile alluxio-s3 --endpoint "http://localhost:29998/" s3api head-object \
  --bucket=testbucket --key=test.txt
{
    "LastModified": "2022-05-03T18:55:01+00:00",
    "ContentLength": 27040,
    "ETag": "0cc175b9c0f1b6a831c399e269772661",
    "ContentType": "application/octet-stream",
    "Metadata": {}
}

$ curl -i -H "Authorization: AWS4-HMAC-SHA256 Credential=testuser/... SignedHeaders=... Signature=..." \
  --head http://localhost:29998/testbucket/test.txt
HTTP/1.1 200 OK
Date: Tue, 03 May 2022 21:54:22 GMT
Last-Modified: Tue, 03 May 2022 21:47:36 GMT
ETag: 0cc175b9c0f1b6a831c399e269772661
Content-Type: application/octet-stream
Content-Length: 27040
Server: Jetty(9.4.43.v20210629)

ListBuckets

AWS CLI
REST Clients

$ aws --profile alluxio-s3 --endpoint "http://localhost:29998/" s3api list-buckets
{
    "Buckets": [
        {
            "Name": "testbucket",
            "CreationDate": "2022-05-03T11:32:34.156000+00:00"
        }
    ]
}

$ curl -i -H "Authorization: AWS4-HMAC-SHA256 Credential=testuser/... SignedHeaders=... Signature=..." \
  -X GET http://localhost:29998/
HTTP/1.1 200 OK
Date: Tue, 03 May 2022 21:35:23 GMT
Content-Type: application/xml
Content-Length: 161
Server: Jetty(9.4.43.v20210629)

<ListAllMyBucketsResult>
  <Buckets>
    <Bucket>
      <Name>testbucket</Name>
      <CreationDate>2022-05-03T14:34:56.420Z</CreationDate>
    </Bucket>
  </Buckets>
</ListAllMyBucketsResult>

ListObjects

AWS CLI
REST Clients

$ aws --profile alluxio-s3 --endpoint "http://localhost:29998/" s3api list-objects \
  --bucket=testbucket
{
    "Contents": [
        {
            "Key": "test.txt",
            "LastModified": "2022-05-03T11:35:59.243000+00:00",
            "Size": 27040
        },
        {
            "Key": "test_copy.txt",
            "LastModified": "2022-05-03T11:37:16.185000+00:00",
            "Size": 27040
        }
    ]
}

$ curl -i -H "Authorization: AWS4-HMAC-SHA256 Credential=testuser/... SignedHeaders=... Signature=..." \
  -X GET http://localhost:29998/testbucket
HTTP/1.1 200 OK
Date: Tue, 03 May 2022 21:50:26 GMT
Content-Type: application/xml
Content-Length: 434
Server: Jetty(9.4.43.v20210629)

<ListBucketResult>
  <version2>false</version2>
  <Contents>
    <Key>test.txt</Key>
    <Size>27040</Size>
    <LastModified>2022-05-03T14:47:36.600Z</LastModified>
  </Contents>
  <Contents>
    <Key>test_copy.txt</Key>
    <Size>27040</Size>
    <LastModified>2022-05-03T14:50:07.790Z</LastModified>
  </Contents>
  <Marker/>
  <IsTruncated>false</IsTruncated>
  <Prefix/>
  <Name>testbucket</Name>
  <MaxKeys>1000</MaxKeys>
  <EncodingType>url</EncodingType>
</ListBucketResult>

ListObjectsV2

AWS CLI
REST Clients

$ aws --profile alluxio-s3 --endpoint "http://localhost:29998/" s3api list-objects-v2 \
  --bucket=testbucket
{
    "Contents": [
        {
            "Key": "test.txt",
            "LastModified": "2022-05-03T11:35:59.243000+00:00",
            "Size": 27040
        },
        {
            "Key": "test_copy.txt",
            "LastModified": "2022-05-03T11:37:16.185000+00:00",
            "Size": 27040
        }
    ]
}

$ curl -i -H "Authorization: AWS4-HMAC-SHA256 Credential=testuser/... SignedHeaders=... Signature=..." \
  -X GET "http://localhost:29998/testbucket?list-type=2"
HTTP/1.1 200 OK
Date: Tue, 03 May 2022 21:56:20 GMT
Content-Type: application/xml
Content-Length: 438
Server: Jetty(9.4.43.v20210629)

<ListBucketResult>
  <version2>true</version2>
  <Contents>
    <Key>test.txt</Key>
    <Size>27040</Size>
    <LastModified>2022-05-03T14:47:36.600Z</LastModified>
  </Contents>
  <Contents>
    <Key>test_copy.txt</Key>
    <Size>27040</Size>
    <LastModified>2022-05-03T14:50:07.790Z</LastModified>
  </Contents>
  <IsTruncated>false</IsTruncated>
  <Prefix/>
  <KeyCount>2</KeyCount>
  <Name>testbucket</Name>
  <MaxKeys>1000</MaxKeys>
  <EncodingType>url</EncodingType>
</ListBucketResult>

PutBucketTagging

AWS CLI
REST Clients

$ aws --profile alluxio-s3 --endpoint "http://localhost:29998/" s3api get-bucket-tagging \
  --bucket=testbucket
{
    "TagSet": []
}

$ aws --profile alluxio-s3 --endpoint "http://localhost:29998/" s3api put-bucket-tagging \
  --bucket=testbucket --tagging='TagSet=[{Key=key1,Value=val1},{Key=key2,Value=val2}]'

$ aws --profile alluxio-s3 --endpoint "http://localhost:29998/" s3api get-bucket-tagging \
  --bucket=testbucket
{
    "TagSet": [
        {
            "Key": "key1",
            "Value": "val1"
        },
        {
            "Key": "key2",
            "Value": "val2"
        }
    ]
}

$ curl -i -H "Authorization: AWS4-HMAC-SHA256 Credential=testuser/... SignedHeaders=... Signature=..." \
  -X GET "http://localhost:29998/testbucket?tagging"
HTTP/1.1 200 OK
Date: Tue, 03 May 2022 23:30:25 GMT
Content-Type: application/xml
Content-Length: 28
Server: Jetty(9.4.43.v20210629)

<Tagging><TagSet/></Tagging>

$ cat tags.xml
<Tagging>
  <TagSet>
    <Tag>
      <Key>key1</Key>
      <Value>val1</Value>
    </Tag>
    <Tag>
      <Key>key2</Key>
      <Value>val2</Value>
    </Tag>
  </TagSet>
</Tagging>

$ curl -i -H "Authorization: AWS4-HMAC-SHA256 Credential=testuser/... SignedHeaders=... Signature=..." \
  -H "Content-Type: application/xml" \
  -X PUT "http://localhost:29998/testbucket?tagging" --data-binary "@tags.xml"
HTTP/1.1 200 OK
Date: Tue, 03 May 2022 23:31:05 GMT
Content-Length: 0
Server: Jetty(9.4.43.v20210629)

$ curl -i -H "Authorization: AWS4-HMAC-SHA256 Credential=testuser/... SignedHeaders=... Signature=..." \
  -X GET "http://localhost:29998/testbucket?tagging"
HTTP/1.1 200 OK
Date: Tue, 03 May 2022 23:31:07 GMT
Content-Type: application/xml
Content-Length: 124
Server: Jetty(9.4.43.v20210629)

<Tagging>
  <TagSet>
    <Tag>
      <Key>key1</Key>
      <Value>val1</Value>
    </Tag>
    <Tag>
      <Key>key2</Key>
      <Value>val2</Value>
    </Tag>
  </TagSet>
</Tagging>

PutObject

AWS CLI
REST Clients

$ aws --profile alluxio-s3 --endpoint "http://localhost:29998/" s3api put-object \
  --bucket=testbucket --key=test.txt --body="${ALLUXIO_HOME}/LICENSE"
{
    "ETag": "911df44b7ff57801ca8d74568e4ebfbe"
}

$ aws --profile alluxio-s3 --endpoint "http://localhost:29998/" s3api list-objects \
  --bucket=testbucket
{
    "Contents": [
        {
            "Key": "test.txt",
            "LastModified": "2022-05-03T11:35:59.243000+00:00",
            "Size": 27040
        }
    ]
}

$ curl -i -H "Authorization: AWS4-HMAC-SHA256 Credential=testuser/... SignedHeaders=... Signature=..." \
  -X PUT http://localhost:29998/testbucket/test.txt -T "${ALLUXIO_HOME}/LICENSE"
HTTP/1.1 100 Continue

HTTP/1.1 200 OK
Date: Tue, 03 May 2022 21:47:36 GMT
ETag: 911df44b7ff57801ca8d74568e4ebfbe
Content-Length: 0
Server: Jetty(9.4.43.v20210629)

$ curl -i -H "Authorization: AWS4-HMAC-SHA256 Credential=testuser/... SignedHeaders=... Signature=..." \
  -X GET http://localhost:29998/testbucket
HTTP/1.1 200 OK
Date: Tue, 03 May 2022 21:47:44 GMT
Content-Type: application/xml
Content-Length: 318
Server: Jetty(9.4.43.v20210629)

<ListBucketResult>
  <version2>false</version2>
  <Contents>
    <Key>test.txt</Key>
    <Size>27040</Size>
    <LastModified>2022-05-03T14:47:36.600Z</LastModified>
  </Contents>
  <Marker/>
  <IsTruncated>false</IsTruncated>
  <Prefix/>
  <Name>testbucket</Name>
  <MaxKeys>1000</MaxKeys>
  <EncodingType>url</EncodingType>
</ListBucketResult>

PutObjectTagging

AWS CLI
REST Clients

$ aws --profile alluxio-s3 --endpoint "http://localhost:29998/" s3api get-object-tagging \
  --bucket=testbucket --key=test.txt
{
    "TagSet": []
}

$ aws --profile alluxio-s3 --endpoint "http://localhost:29998/" s3api put-object-tagging \
  --bucket=testbucket --key=test.txt --tagging='TagSet=[{Key=key1,Value=val1},{Key=key2,Value=val2}]'

$ aws --profile alluxio-s3 --endpoint "http://localhost:29998/" s3api get-object-tagging \
  --bucket=testbucket --key=test.txt
{
    "TagSet": [
        {
            "Key": "key1",
            "Value": "val1"
        },
        {
            "Key": "key2",
            "Value": "val2"
        }
    ]
}

$ curl -i -H "Authorization: AWS4-HMAC-SHA256 Credential=testuser/... SignedHeaders=... Signature=..." \
  -X GET "http://localhost:29998/testbucket/test.txt?tagging"
HTTP/1.1 200 OK
Date: Tue, 03 May 2022 23:30:25 GMT
Content-Type: application/xml
Content-Length: 28
Server: Jetty(9.4.43.v20210629)

<Tagging><TagSet/></Tagging>

$ cat tags.xml
<Tagging>
  <TagSet>
    <Tag>
      <Key>key1</Key>
      <Value>val1</Value>
    </Tag>
    <Tag>
      <Key>key2</Key>
      <Value>val2</Value>
    </Tag>
  </TagSet>
</Tagging>

$ curl -i -H "Authorization: AWS4-HMAC-SHA256 Credential=testuser/... SignedHeaders=... Signature=..." \
  -H "Content-Type: application/xml" \
  -X PUT "http://localhost:29998/testbucket/test.txt?tagging" --data-binary "@tags.xml"
HTTP/1.1 200 OK
Date: Tue, 03 May 2022 23:35:28 GMT
Content-Length: 0
Server: Jetty(9.4.43.v20210629)

$ curl -i -H "Authorization: AWS4-HMAC-SHA256 Credential=testuser/... SignedHeaders=... Signature=..." \
  -X GET "http://localhost:29998/testbucket/test.txt?tagging"
HTTP/1.1 200 OK
Date: Tue, 03 May 2022 23:35:58 GMT
Content-Type: application/octet-stream
Content-Length: 126
Server: Jetty(9.4.43.v20210629)

<Tagging>
  <TagSet>
    <Tag>
      <Key>key1</Key>
      <Value>val1</Value>
    </Tag>
    <Tag>
      <Key>key2</Key>
      <Value>val2</Value>
    </Tag>
  </TagSet>
</Tagging>

Categories

S3 API

使用示例

S3 API 操作