hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Steve Loughran (JIRA)" <j...@apache.org>
Subject [jira] [Created] (HADOOP-15650) Add custom InstanceProfileCredentialsProvider with more resilience to throttling
Date Fri, 03 Aug 2018 00:17:00 GMT
Steve Loughran created HADOOP-15650:
---------------------------------------

             Summary: Add custom InstanceProfileCredentialsProvider with more resilience to
throttling
                 Key: HADOOP-15650
                 URL: https://issues.apache.org/jira/browse/HADOOP-15650
             Project: Hadoop Common
          Issue Type: Sub-task
          Components: fs/s3
    Affects Versions: 3.1.0
            Reporter: Steve Loughran


Add our own InstanceProfileCredentialsProvider class which uses the AWS implementation to
retrieve credentials from EC2's instance info, but more resilient to overloading.

# pass in client config with retry logic (HADOOP-15603)
# use Invoke.retry() to retry
# log/measure failures
# maybe use the Async feature of the AWS SDK class, so that credential renewer doesn't block
IO.
# be shared amongst all AWS auth chains which need these credentials.

The singleton we current use for IAM auth doesn't do async, which is good as it ensures that
we don't prematurely close it when {{AWSCredentialProviderList.close()}} closes its children.




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

---------------------------------------------------------------------
To unsubscribe, e-mail: common-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-dev-help@hadoop.apache.org


Mime
View raw message