hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Owen O'Malley (JIRA)" <j...@apache.org>
Subject [jira] Updated: (HADOOP-597) transmission errors to the reduce will cause map output to be considered lost
Date Wed, 11 Oct 2006 17:22:37 GMT
     [ http://issues.apache.org/jira/browse/HADOOP-597?page=all ]

Owen O'Malley updated HADOOP-597:
---------------------------------

    Attachment: map-output-protect.patch

Here is a patch that fixes the problem. I use a flag to mark that the fault was on the input
and therefore the map output should be declared lost. I'm not really that happy with all of
the catches that rethrow, but I didn't see a more straightforward way of getting the right
effect.

> transmission errors to the reduce will cause map output to be considered lost
> -----------------------------------------------------------------------------
>
>                 Key: HADOOP-597
>                 URL: http://issues.apache.org/jira/browse/HADOOP-597
>             Project: Hadoop
>          Issue Type: Bug
>    Affects Versions: 0.7.0
>            Reporter: Owen O'Malley
>         Assigned To: Owen O'Malley
>             Fix For: 0.7.1
>
>         Attachments: map-output-protect.patch
>
>
> When the mapOutput servlet gets an IOException from reading the map output file, it correctly
declares the map output lost. However, an IOException writing to the socket should NOT cause
the map output to be declared lost. Unfortunately, the current implementation always declares
the map output lost. When the map output is lost, the map is re-run on a different computer.

-- 
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message