hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Steve Loughran (JIRA)" <j...@apache.org>
Subject [jira] [Comment Edited] (HADOOP-14556) S3A to support Delegation Tokens
Date Fri, 04 Aug 2017 19:32:00 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-14556?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16114868#comment-16114868
] 

Steve Loughran edited comment on HADOOP-14556 at 8/4/17 7:31 PM:
-----------------------------------------------------------------

Patch 001, as a PoC rather than anything ready for line by line review

This adds the option of enabling delegation tokens. If set, when logged in with full credentials,
then DTs can be obtained for a limited (configurable) lifespan...these are just session credentials.
The DT can be marshalled over the wire then unmarshalled, where the new {{DelegationTokenCredentialProvider}}
provider will find them and use them for auth. The encryption settings are also passed in,
so a DT can contain the key for 

These tokens cannot be renewed, nor can they be revoked. But they will always have limited
life. 

* Minimal changes to S3AFS, as the new work is all in {{S3ADelegationTokens}} apart from the
init logic and binding to the {{getCanonicalServiceName()}},
 and {{getDelegationToken()}} methods. As long as the tokens also work with DDB then s3guard
integration should be straightforward.
* bits of refactoring about how session credentials are extracted and validated in our AWS
credential providers.
* There's no support for propagating encryption options & key secrets; that's the obvious
next step. A user should be able to provide their own key for file access, which would then
be marshalled over to the workers.

+ does some reinstatement of the URI, HADOOP-14723, but that should really be isolated





was (Author: stevel@apache.org):
Patch 001, as a PoC rather than anything ready for line by line review

This adds the option of enabling delegation tokens. If set, when logged in with full credentials,
then DTs can be obtained for a limited (configurable) lifespan...these are just session credentials.
The DT can be marshalled over the wire then unmarshalled, where the new {{DelegationTokenCredentialProvider}}
provider will find them and use them for auth. The encryption settings are also passed in,
so a DT can contain the key for 

These tokens cannot be renewed, nor can they be revoked. But they will always have limited
life. 

* Minimal changes to S3AFS, as the new work is all in {{S3ADelegationTokens}} apart from the
init logic and binding to the {{ getCanonicalServiceName() }},
 and {{getDelegationToken()}} methods. As long as the tokens also work with DDB then s3guard
integration should be straightforward.
* bits of refactoring about how session credentials are extracted and validated in our AWS
credential providers.
* There's no support for propagating encryption options & key secrets; that's the obvious
next step. A user should be able to provide their own key for file access, which would then
be marshalled over to the workers.

+ does some reinstatement of the URI, HADOOP-14723, but that should really be isolated




> S3A to support Delegation Tokens
> --------------------------------
>
>                 Key: HADOOP-14556
>                 URL: https://issues.apache.org/jira/browse/HADOOP-14556
>             Project: Hadoop Common
>          Issue Type: Sub-task
>          Components: fs/s3
>    Affects Versions: 2.8.1
>            Reporter: Steve Loughran
>            Assignee: Steve Loughran
>         Attachments: HADOOP-14556-001.patch
>
>
> S3A to support delegation tokens where
> * an authenticated client can request a token via {{FileSystem.getDelegationToken()}}
> * Amazon's token service is used to request short-lived session secret & id; these
will be saved in the token and  marshalled with jobs
> * A new authentication provider will look for a token for the current user and authenticate
the user if found
> This will not support renewals; the lifespan of a token will be limited to the initial
duration. Also, as you can't request an STS token from a temporary session, IAM instances
won't be able to issue tokens.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org


Mime
View raw message