hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Steve Loughran (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HADOOP-12844) Recover when S3A fails on IOException in read()
Date Fri, 22 Apr 2016 11:44:13 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-12844?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15253785#comment-15253785
] 

Steve Loughran commented on HADOOP-12844:
-----------------------------------------

I've taken this over. It's actually simpler than this patch, as all that's needed is the existing
clause catching some socket exceptions to be expanded to catch any IOE, and then log and retry
... this is common code which can be shared in both read operations.

what is important is to catch and respond to EOF exceptions before the generic IOE clause

> Recover when S3A fails on IOException in read()
> -----------------------------------------------
>
>                 Key: HADOOP-12844
>                 URL: https://issues.apache.org/jira/browse/HADOOP-12844
>             Project: Hadoop Common
>          Issue Type: Sub-task
>          Components: fs/s3
>    Affects Versions: 2.7.1, 2.7.2
>            Reporter: Pieter Reuse
>            Assignee: Pieter Reuse
>         Attachments: HADOOP-12844.001.patch
>
>
> This simple patch catches IOExceptions in S3AInputStream.read(byte[] buf, int off, int
len) and reopens the connection on the same location as it was before the exception.
> This is similar to the functionality introduced in S3N in [HADOOP-6254|https://issues.apache.org/jira/browse/HADOOP-6254],
for exactly the same reason.
> Patch developed in cooperation with [~emres].



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message