hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Mahadev konar (JIRA)" <j...@apache.org>
Subject [jira] Updated: (HADOOP-343) In case of dead task tracker, the copy mapouts try copying all mapoutputs from this tasktracker
Date Wed, 05 Jul 2006 18:21:30 GMT
     [ http://issues.apache.org/jira/browse/HADOOP-343?page=all ]

Mahadev konar updated HADOOP-343:
---------------------------------

    Attachment: bugfix.patch

This patch updates its mapoutput locations in case of any failures on copying from the task
tracker. So, in case a copy failed from a task tracker, the map outputs corresponding to this
node will be marked stale. The ReduceTask will ask for the mapout locations again for these
stale map outputs. This patch also fixes a bug wherein the tasks keep polling the job tracker
for map otputs "in a loop" without sleeping/waiting. With this fix the tasks will poll evry
MIN_POLL_INTERVAL before querying the job tracker. 

> In case of dead task tracker, the copy mapouts try copying all mapoutputs from this tasktracker
> -----------------------------------------------------------------------------------------------
>
>          Key: HADOOP-343
>          URL: http://issues.apache.org/jira/browse/HADOOP-343
>      Project: Hadoop
>         Type: Bug

>   Components: mapred
>     Versions: 0.5.0
>     Reporter: Mahadev konar
>     Assignee: Mahadev konar
>  Attachments: bugfix.patch
>
> In case of a dead task tracker, the reduces which do not have the updated map out locations
try copygin files from this node and since there are failures on copying, this leads to backoff
and slowing down of the copy pahse.

-- 
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
   http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see:
   http://www.atlassian.com/software/jira


Mime
View raw message