hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Steve Loughran (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HADOOP-15141) Support IAM Assumed roles in S3A
Date Wed, 17 Jan 2018 11:51:00 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-15141?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16328656#comment-16328656

Steve Loughran commented on HADOOP-15141:

Oh, OK. I had just been working on some changes :)

* move the new authenticator to a new patch, s3a.auth
* add a class alongside., "RoleModel" to actually build up the JSON to pump out as valid AWS
role policy
* tests to understand what permissions
* fix to innerDelete() so that if you can't create a mock parent dir marker on a directory
delete, it doesn't trigger a failure

The latter means that If I only have write access to /user/stevel and I delete /user/stevel,
if the attempt to create a /user/ marker fails, the delete still succeeds.

Essentially, I'm adding support into S3A to handle the situation "user doesn't have write
access to everywhere" via test-and-see, using this for the tests, with RoleModel there to
set up the statements & policies properly. I'll no doubt need to play with: rename, MPU
(large files, commit, commit -> abort), s3guard.

Apart from the move to the new package, no other changes to the authenticator itself, that'
s just adding the ability to do the user lockdown

Let me merge mine back in and do a followup


> Support IAM Assumed roles in S3A
> --------------------------------
>                 Key: HADOOP-15141
>                 URL: https://issues.apache.org/jira/browse/HADOOP-15141
>             Project: Hadoop Common
>          Issue Type: Sub-task
>          Components: fs/s3
>    Affects Versions: 3.0.0
>            Reporter: Steve Loughran
>            Assignee: Steve Loughran
>            Priority: Major
>             Fix For: 3.1.0
>         Attachments: HADOOP-15141-001.patch, HADOOP-15141-002.patch, HADOOP-15141-003.patch,
HADOOP-15141-004.patch, HADOOP-15141-005.patch, HADOOP-15141-006.patch
> Add the ability to use assumed roles in S3A
> * Add a property fs.s3a.assumed.role.arn for the ARN of the assumed role
> * add a new provider which grabs that and other properties and then creates a {{STSAssumeRoleSessionCredentialsProvider}}
from it.
> * This also needs to support building up its own list of aws credential  providers, from
a different property; make the changes to S3AUtils for that
> * Tests
> * docs
> * and have the AwsProviderList forward closeable to it.
> * Get picked up automatically by DDB/s3guard

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org

View raw message