hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hudson (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HADOOP-13130) s3a failures can surface as RTEs, not IOEs
Date Sat, 21 May 2016 15:57:12 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-13130?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15295092#comment-15295092
] 

Hudson commented on HADOOP-13130:
---------------------------------

SUCCESS: Integrated in Hadoop-trunk-Commit #9836 (See [https://builds.apache.org/job/Hadoop-trunk-Commit/9836/])
HADOOP-13130. s3a failures can surface as RTEs, not IOEs. (Steve (stevel: rev 39ec1515a205952eda7e171408a8b83eceb4abde)
* hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/InvalidRequestException.java
* hadoop-tools/hadoop-aws/src/test/java/org/apache/hadoop/fs/s3a/TestS3AAWSCredentialsProvider.java
* hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/PathNotFoundException.java
* hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/AWSClientIOException.java
* hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/S3AFastOutputStream.java
* hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/PathPermissionException.java
* hadoop-tools/hadoop-aws/src/test/java/org/apache/hadoop/fs/s3a/TestS3AConfiguration.java
* hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/S3AInputStream.java
* hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/S3AFileSystem.java
* hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/package-info.java
* hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/PathAccessDeniedException.java
* hadoop-tools/hadoop-aws/src/test/java/org/apache/hadoop/fs/s3a/TestS3AFailureHandling.java
* hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/S3AOutputStream.java
* hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/S3AUtils.java
* hadoop-tools/hadoop-aws/src/test/java/org/apache/hadoop/fs/s3a/S3ATestUtils.java
* hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/AWSS3IOException.java
* hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/AWSServiceIOException.java


> s3a failures can surface as RTEs, not IOEs
> ------------------------------------------
>
>                 Key: HADOOP-13130
>                 URL: https://issues.apache.org/jira/browse/HADOOP-13130
>             Project: Hadoop Common
>          Issue Type: Sub-task
>          Components: fs/s3
>    Affects Versions: 2.7.2
>            Reporter: Steve Loughran
>            Assignee: Steve Loughran
>             Fix For: 2.8.0
>
>         Attachments: HADOOP-13130-001.patch, HADOOP-13130-002.patch, HADOOP-13130-002.patch,
HADOOP-13130-003.patch, HADOOP-13130-004.patch, HADOOP-13130-005.patch, HADOOP-13130-016.patch,
HADOOP-13130-branch-2-006.patch, HADOOP-13130-branch-2-007.patch, HADOOP-13130-branch-2-008.patch,
HADOOP-13130-branch-2-009.patch, HADOOP-13130-branch-2-010.patch, HADOOP-13130-branch-2-011.patch,
HADOOP-13130-branch-2-012.patch, HADOOP-13130-branch-2-013.patch, HADOOP-13130-branch-2-014.patch,
HADOOP-13130-branch-2-015.patch, HADOOP-13130-branch-2-016.patch
>
>
> S3A failures happening in the AWS library surface as {{AmazonClientException}} derivatives,
rather than IOEs. As the amazon exceptions are runtime exceptions, any code which catches
IOEs for error handling breaks.
> The fix will be to catch and wrap. The hard thing will be to wrap it with meaningful
exceptions rather than a generic IOE. Furthermore, if anyone has been catching AWS exceptions,
they are going to be disappointed. That means that fixing this situation could be considered
"incompatible" —but only for code which contains assumptions about the underlying FS and
the exceptions they raise.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org


Mime
View raw message