hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Owen O'Malley (JIRA)" <j...@apache.org>
Subject [jira] Updated: (HADOOP-1183) MapTask completion not recorded properly at the Reducer's end
Date Mon, 02 Apr 2007 23:15:33 GMT

     [ https://issues.apache.org/jira/browse/HADOOP-1183?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel

Owen O'Malley updated HADOOP-1183:

    Fix Version/s:     (was: 0.12.3)
           Status: Open  (was: Patch Available)

I'm uneasy about this patch. The underlying code is very complex, the patch is adding substantial
complexity, and it isn't clear to me that this is the right direction. I think we should post-pone
this fix and likely redesign the fetcher in 0.13.

One possible approach to simplifying this section of code would be to make an array of states
for each of the map outputs (INITIAL, LOCATED, FETCHING, DONE, FAILED) and process the map
outputs using a DFA. Another structure that might make sense is an array of the best MapOutputLocation
for each map.


> MapTask completion not recorded properly at the Reducer's end
> -------------------------------------------------------------
>                 Key: HADOOP-1183
>                 URL: https://issues.apache.org/jira/browse/HADOOP-1183
>             Project: Hadoop
>          Issue Type: Bug
>          Components: mapred
>    Affects Versions: 0.12.2
>            Reporter: Devaraj Das
>         Assigned To: Devaraj Das
>            Priority: Critical
>             Fix For: 0.13.0
>         Attachments: 1183.new.patch, 1183.new1.patch, 1183.patch
> A couple of reducers were continuously trying to fetch map outputs from a lost tasktracker.
Although the tasks running on that lost TT successfully reexecuted elsewhere, the Reducers'
tasktrackers didn't correctly note those events.

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message