hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Mingliang Liu (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (HADOOP-14248) Retire SharedInstanceProfileCredentialsProvider after AWS SDK upgrade
Date Tue, 28 Mar 2017 21:02:41 GMT

     [ https://issues.apache.org/jira/browse/HADOOP-14248?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel

Mingliang Liu updated HADOOP-14248:
    Attachment: HADOOP-14248-branch-2.001.patch

Thanks [~stevel@apache.org] for your comments.

I think it's OK to remove this class in next major version, aka 2.x -> 3.x. It still merits
a lot to keep this in 2.x branches in case people have started using it (that is a really
meaningful improvement). We're kind enough to warn people about this in branch-2.9, where
the AWS SDK got upgraded. {{branch-2.8}} will stay as it is. I attach the branch-2 patch.
Does it make sense?

branch-2 patch was tested against us-west-1 on my laptop (using secret credentials) and AWS
EC2 (using instance profile credentials).

> Retire SharedInstanceProfileCredentialsProvider after AWS SDK upgrade
> ---------------------------------------------------------------------
>                 Key: HADOOP-14248
>                 URL: https://issues.apache.org/jira/browse/HADOOP-14248
>             Project: Hadoop Common
>          Issue Type: Sub-task
>          Components: fs/s3
>    Affects Versions: 3.0.0-alpha3
>            Reporter: Mingliang Liu
>            Assignee: Mingliang Liu
>         Attachments: HADOOP-14248.000.patch, HADOOP-14248-branch-2.001.patch
> This is from the discussion in [HADOOP-13050].
> So [HADOOP-13727] added the SharedInstanceProfileCredentialsProvider, which effectively
reduces high number of connections to EC2 Instance Metadata Service caused by InstanceProfileCredentialsProvider.
That patch, in order to prevent the throttling problem, defined new class {{SharedInstanceProfileCredentialsProvider}}
as a subclass of {{InstanceProfileCredentialsProvider}}, which enforces creation of only a
single instance.
> Per [HADOOP-13050], we upgraded the AWS Java SDK. Since then, the {{InstanceProfileCredentialsProvider}}
in SDK code internally enforces a singleton. That  confirms that our effort in [HADOOP-13727]
makes 100% sense. Meanwhile, {{SharedInstanceProfileCredentialsProvider}} can retire gracefully
in trunk branch.

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org

View raw message