hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Kazuyuki Tanimura (JIRA)" <j...@apache.org>
Subject [jira] [Comment Edited] (HADOOP-14237) S3A Support Shared Instance Profile Credentials Across All Hadoop Nodes
Date Sat, 15 Apr 2017 00:53:41 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-14237?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15969742#comment-15969742

Kazuyuki Tanimura edited comment on HADOOP-14237 at 4/15/17 12:53 AM:

I will make this an independent credential provider.

was (Author: kazuyukitanimura):
I will make this an independnet credential provider.

> S3A Support Shared Instance Profile Credentials Across All Hadoop Nodes
> -----------------------------------------------------------------------
>                 Key: HADOOP-14237
>                 URL: https://issues.apache.org/jira/browse/HADOOP-14237
>             Project: Hadoop Common
>          Issue Type: Sub-task
>          Components: fs/s3
>    Affects Versions: 2.8.0, 3.0.0-alpha1, 3.0.0-alpha2, 2.8.1
>         Environment: EC2, AWS
>            Reporter: Kazuyuki Tanimura
>            Assignee: Kazuyuki Tanimura
> When I run a large Hadoop cluster on EC2 instances with IAM Role, it fails getting the
instance profile credentials, eventually all jobs on the cluster fail. Since a number of S3A
clients (all mappers and reducers) try to get the credentials, the AWS credential endpoint
starts responding 5xx and 4xx error codes.
> SharedInstanceProfileCredentialsProvider.java is sort of trying to solve it, but it still
does not share the credentials with other EC2 nodes / JVM processes.
> This issue prevents users from creating Hadoop clusters on EC2

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org

View raw message