S3 Third-Party Authentication

Slack Docker Pulls

Overview

Alluxio adapts the OIDC protocol to handle third-party authentication for Alluxio S3 API. OIDC Providers that are supported in Alluxio must have their HTTPS public JSON WebKey Set(JWKS) endpoint which will return the public key to verify standard JWT Token. Now Alluxio supports integration with a few OIDC providers, including PingFederate and Amazon Cognito.

If Alluxio Third-party Authentication is enabled,

  • Alluxio validate the token provided from the third-party identity management system(such as PingFederate).
  • Alluxio provide temporary security credential for users who have been authenticated with a OIDC provider.
  • Alluxio authenticate the interaction between HTTPs client and S3 Rest Proxy Server.
  • Each temporary security credential is valid for a particular duration. when the duration end, the temporary credential will expire.

Configuration

To enable third-party authentication for S3 API, the following configuration properties in conf/alluxio-site.properties must be set:

alluxio.s3.rest.authentication.oidc.enabled=true
alluxio.s3.rest.authentication.jwksaddr=<OIDC JWKs endpoint address>

<OIDC JWKs endpoint address> is the HTTPS url address to retrieve and cache keys for OIDC Token. When alluxio enable authentication through OIDC Token, this parameter must be set with the JWKs endpoint address provided by OIDC Provider. It will be used to verify the ID token from OIDC Provider.

Note: When you enable OIDC Authentication for Alluxio S3 API, you need to set up TLS encryption in Alluxio, see TLS Encryption for Network Communication.

If you have a pre-defined user field in token, and you want to assume it as the username when AssumeRoleWithWebIdentity, the following configuration properties in conf/alluxio-site.properties must be set:

alluxio.s3.rest.authentication.assume.role.from.token=true
alluxio.s3.rest.authentication.assume.role.token.field=<custom_field_in_token>

<custom_field_in_token> is the name of the specific field key in the token.

Requesting temporary security credentials

You can call the AssumeRoleWithWebIdentity action of Alluxio STS to get a set of temporary security credentials for users who have been authenticated with a OIDC provider.

When you make this call, you pass the following information:

  • WebIdentityToken which is the token that the app gets from the IDP after the app authenticates the user.
  • DurationSeconds (Optional) which specifies the duration of the temporary security credentials. Use the DurationSeconds parameter to specify the duration of the role session from 900 seconds (15 minutes) up to the 43200 seconds (12 hours). If you do not pass this parameter, the temporary credentials expire in one hour.
  • RoleSessionName which is the identifier for the assumed role session. The temporary security credentials that your application will use are associated with that user. By default, it will be associated with the user in Alluxio.

    Note: If you want to use a specific filed value in the token as the username, you need to set alluxio.s3.rest.authentication.assume.role.from.token=true and alluxio.s3.rest.authentication.assume.role.token.field=<field_name>.

When you call AssumeRoleWithWebIdentity, Alluxio verifies the authenticity of the token. For example, Alluxio might make a call to the provider to get the JWKs info and verify the token that the app has passed. After Alluxio validates the token, it will return the following information to you:

  • <Credentials> which consist of an access key ID, a secret access key, and a session token.
  • <SubjectFromWebIdentityToken> which contains the unique user ID.
  • <Audience> which represent intended audience of the web identity token. This is traditionally the client identifier issued to the application that requested the web identity token.

When you have the temporary security credentials, you can use them to make S3 API calls. The difference is that you must include the session token, which lets Alluxio verify that the temporary security credentials are valid.

Your app should cache the credentials. As noted, by default the credentials expire after an hour. It’s up to you and your app to decide when to call AssumeRoleWithWebIdentity again. Call this operation to get a new set of temporary security credentials before the old ones expire.

Request Example

https://xx.xx.xx:39999/api/v1/sts
?Action=AssumeRoleWithWebIdentity
&DurationSeconds=3600
&ProviderId=www.alluxio.com
&RoleSessionName=app1
&WebIdentityToken=eyJhbGciOiJSUzI1NiIsImtpZCI6ImRlZmF1bHQifQ.eyJpc3MiOiJodHRwczovL2F1dGgucGluZ29uZS5jb20vNTEyMTc0ZGYtYz
ZjNi00NDdjLTk2NTctYTY1OWM2ODU1OWQ0L2FzIiwic3ViIjoiNjVkODdiNWUtMjJmZC00YWJmLWJhNTItZjE2NmU2ZGUxNDI3IiwiYXVkIjoiZjVlOGVlN
2EtMDkwMi00ZDg5LTk0MjItMTM1MGVhZDg3ODYxIiwiaWF0IjoxNjY1Mzk1NzE2LCJleHAiOjE2NjUzOTkzMTYsImFjciI6IlNpbmdsZV9GYWN0b3IiLCJh
bXIiOlsicHdkIl0sImF1dGhfdGltZSI6MTY2NTM5NTcxNSwibm9uY2UiOiIxMjMiLCJzaWQiOiIxOTBhYTZlOS02M2E4LTQyY2MtYjI5ZC1mM2QwOTRjYmU
5OWMiLCJyb2xlIjpbInRlc3QtZ3JvdXAiXSwicHJlZmVycmVkX3VzZXJuYW1lIjoiYXBwX3VzZXJfMSIsImdpdmVuX25hbWUiOiJUZXN0IiwidXBkYXRlZF
9hdCI6MTY2NTM5NTcxNSwiZmFtaWx5X25hbWUiOiJBcHBsaWNhdGlvblVzZXIiLCJlbnYiOiI1MTIxNzRkZi1jNmM2LTQ0N2MtOTY1Ny1hNjU5YzY4NTU5Z
DQiLCJvcmciOiI1NjgzMGRmZC0xMDU2LTQzNTQtYWUzYS1lN2Q3NWRjZTlmMWUiLCJwMS5yZWdpb24iOiJOQSJ9.b-3CdLJRIhJXeMDcKeUlQAF5OCqWn_6
rqaOUrqITlITDtvIQK2CAnLV4Uwk1X8pL_EMwfMpcHymPfDv0qyhuaX4H5fdHgDV2PBwNw5I0-fUPx2czSuPc1GvH2yzDXEKO2l-wMKJmCoB0olPVK0JIbz
pDGvYwsMaeDNVpVj4OUP59SHBAqvxEpwDaUnMJUT-cImAA_H2Om5en3Ztsw-gp8IQSQ23tZpdwVewp-9-xQI_QNRvLTVL3tjWRvb0N5sCZ1qVbEL68VWfu8
fUFUyXlwuXFooQEnrCnL9lLli-PJuI68ZjL7bIMAmwQJFxAzRyK6Vdv8tNux87IsTWte0pVmA

Response Example

<AssumeRoleWithWebIdentityResponse>
  <AssumeRoleWithWebIdentityResult>
    <SubjectFromWebIdentityToken>65d87b5e-22fd-4abf-ba52-f166e6de1427</SubjectFromWebIdentityToken>
    <Audience>f5e8ee7a-0902-4d89-9422-1350ead87861</Audience>
    <Credentials>
      <SessionToken>AQoDYXdzEE0a8ANXXXXXXXXNO1ewxE5TijQyp+IEXAMPLE</SessionToken>
      <SecretAccessKey>wJalrXUtnFEMI/K7MDENG/bPxRfiCYzEXAMPLEKEY</SecretAccessKey>
      <Expiration>2022-09-30T08:17:23Z</Expiration>
      <AccessKeyId>ASgeIAIOSFODNN7EXAMPLE</AccessKeyId>
    </Credentials>
    <SourceIdentity></SourceIdentity>
    <Provider>www.alluxio.com</Provider>
  </AssumeRoleWithWebIdentityResult>
  <ResponseMetadata>
    <RequestId>ad4156e9-bce1-11e2-82e6-6b6efEXAMPLE</RequestId>
  </ResponseMetadata>
</AssumeRoleWithWebIdentityResponse>

Using temporary credentials in S3 API

If you’re making HTTPs S3 API requests to Alluxio, you can use the access key ID, secret access key and session token that you receive from the AssumeRoleWithWebIdentity action. You use the access key ID and secret access key the same way. And You need to add the session token to an HTTP header named x-amz-security-token.

$ curl -i -H "Authorization: AWS4-HMAC-SHA256 Credential=testuser/... SignedHeaders=... Signature=..." -H "x-amz-security-token:eyJhbGciOiJSUzI1NiIsImtpZCI6ImRlZmF1bHQifQ.eyJpc3MiOiJodHRwczovL2F1dGgucGluZ29uZS5jb20vNTEyMTc0ZGYtYzZjNi00" ...