hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Steve Loughran (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HADOOP-12807) S3AFileSystem should read AWS credentials from environment variables
Date Wed, 17 Feb 2016 10:32:18 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-12807?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15150254#comment-15150254

Steve Loughran commented on HADOOP-12807:

I now understand; — and I see it looks like a simple change. But while It may just be a
one-liner, regression testing means that a one line change has to be treated as seriously
as any other, and added to the release process.

And it's going to be hard to test as we can't set env vars inside the test JVM. You are going
to have to 

# document the test process in hadoop-tools/hadoop-aws/src/site/markdown/tools/hadoop-aws/index.md
# follow the contribution instructions in https://wiki.apache.org/hadoop/HowToContribute#Submitting_patches_against_object_stores_such_as_Amazon_S3.2C_OpenStack_Swift_and_Microsoft_Azure

It's not going to get a look at until that process is followed. Jenkins doesn't test against
s3; the extra requirements are there to make sure that whoever submits it has at least run
and passed the tests.

target release would probably be 2.9 + possible backports; please leave out the "fix-version"
until the fix is applied, as that confuses the release notes. Thanks.

> S3AFileSystem should read AWS credentials from environment variables
> --------------------------------------------------------------------
>                 Key: HADOOP-12807
>                 URL: https://issues.apache.org/jira/browse/HADOOP-12807
>             Project: Hadoop Common
>          Issue Type: Improvement
>          Components: fs/s3
>    Affects Versions: 2.7.2
>            Reporter: Tobin Baker
>            Priority: Minor
>         Attachments: HADOOP-12807-1.patch
> Unlike the {{DefaultAWSCredentialsProviderChain}} in the AWS SDK, the {{AWSCredentialsProviderChain}}
constructed by {{S3AFileSystem}} does not include an {{EnvironmentVariableCredentialsProvider}}
instance. This prevents users from supplying AWS credentials in the environment variables
{{AWS_ACCESS_KEY_ID}} and {{AWS_SECRET_ACCESS_KEY}}, which is the only alternative in some
> In my scenario, I need to access S3 from within a test running in a CI environment that
does not support IAM roles but does allow me to supply encrypted environment variables. Thus,
the only secure approach I can use is to supply my AWS credentials in environment variables
(plaintext configuration files are out of the question).

This message was sent by Atlassian JIRA

View raw message