hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Ramya R (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HADOOP-5338) Reduce tasks are stuck waiting for map outputs when none are in progress
Date Wed, 04 Mar 2009 13:43:56 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-5338?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12678732#action_12678732
] 

Ramya R commented on HADOOP-5338:
---------------------------------

Tested the above patch on a 500node cluster and the reducers are no longer stuck and the job
successfully completes after multiple JT restarts.
However, there is one thing to notice. The situation where "reducers wait for maps when none
are running" still occurs. But due to the above patch,this situation is handled in a way that
the reducers don't hang infinitely. Instead the TT pulls back all the events and completes
the task successfully.


> Reduce tasks are stuck waiting for map outputs when none are in progress
> ------------------------------------------------------------------------
>
>                 Key: HADOOP-5338
>                 URL: https://issues.apache.org/jira/browse/HADOOP-5338
>             Project: Hadoop Core
>          Issue Type: Bug
>          Components: mapred
>    Affects Versions: 0.20.0
>            Reporter: Ramya R
>            Assignee: Amar Kamat
>            Priority: Blocker
>             Fix For: 0.20.0
>
>         Attachments: HADOOP-5338-v2.1.patch, log.txt
>
>
> When JT is restarted several times, a situation is encountered when the reduce tasks
are stuck forever waiting for map outputs. However 100%map is complete and none of the map
tasks are in progress. The reduce tasks wait infinitely.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message