hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Devaraj Das (JIRA)" <j...@apache.org>
Subject [jira] Updated: (HADOOP-1077) Race condition in fetching map outputs (might lead to hung reduces)
Date Wed, 07 Mar 2007 14:53:24 GMT

     [ https://issues.apache.org/jira/browse/HADOOP-1077?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Devaraj Das updated HADOOP-1077:
--------------------------------

    Attachment: 1077.patch

This patch does the following:
1) Adds a new hashmap to maintain the fetches-in-progress. This is used to track fetches and
to take of the problem where we receive a new map output location for a given mapId while
we are fetching (and will probably fail) the output for that same mapId from some other location.
2) An entry for a mapId is made in the fetchInProgress map as soon as we receive a map output
location from the JobTracker if the neededOutputs has the mapId. The entry for the mapId is
not deleted until we have successfully copied the output. 
3) During the time the output is copied, it might so happen that the TT is lost and we come
to know about it after a while (after connect times out, etc.). In the meanwhile, if another
execution of the lost task happened somewhere, we get the event for that and schedule that
fetch as well. The output of first successful copier is considered as valid and the other
becomes obsolete. In the existing code, this second event would be lost since we depended
only on the neededOutputs list (from where entries are removed as soon as the fetches are
scheduled). Now an additional check is done to see whether the mapId exists in fetchesInProgress.

> Race condition in fetching map outputs (might lead to hung reduces)
> -------------------------------------------------------------------
>
>                 Key: HADOOP-1077
>                 URL: https://issues.apache.org/jira/browse/HADOOP-1077
>             Project: Hadoop
>          Issue Type: Bug
>          Components: mapred
>            Reporter: Devaraj Das
>         Assigned To: Devaraj Das
>         Attachments: 1077.patch
>
>
> Sometimes when a map task is lost while the map-output fetch is happening from the TT
for that task, and the lost map has successfully executed on some other node, the event for
that successful execution is lost at the fetching TT. The fetching TT might eventually fail
to fetch the output for the lost task, but then since the event for the new run of the lost
map might also have been lost, the fetching TT might hang.
> This "hung" problem was discovered while working on HADOOP-1060.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message