hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Andrew Hitchcock (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HADOOP-6254) s3n fails with SocketTimeoutException
Date Mon, 14 Sep 2009 22:05:57 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-6254?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12755218#action_12755218
] 

Andrew Hitchcock commented on HADOOP-6254:
------------------------------------------

I don't believe the patch sets an infinite timeout. Upon reopening the connection, it tries
once and if that fails then the call fails. It doesn't recurse.

I think this is better than fiddling with the timeout because now users don't have to worry
about adjusting another setting, this should just work out of the box. In my testing, I was
able to reproduce the SocketTimeoutException and this patch reliably fixed it.

> s3n fails with SocketTimeoutException
> -------------------------------------
>
>                 Key: HADOOP-6254
>                 URL: https://issues.apache.org/jira/browse/HADOOP-6254
>             Project: Hadoop Common
>          Issue Type: Bug
>          Components: fs/s3
>    Affects Versions: 0.18.3, 0.19.2, 0.20.1
>            Reporter: Andrew Hitchcock
>            Assignee: Andrew Hitchcock
>         Attachments: HADOOP-6254.diff
>
>
> If a user's map function is CPU intensive and doesn't read from the input very quickly,
compounded by the buffering of input, then S3 might think the connection has been lost and
will close the connection. Then when the user attempts to read from the input again, they'll
receive a SocketTimeoutException and the task will fail.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message