hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Runping Qi (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HADOOP-5289) Tasks are not scheduled even though task trackers have extra slots
Date Thu, 19 Feb 2009 20:02:02 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-5289?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12675111#action_12675111
] 

Runping Qi commented on HADOOP-5289:
------------------------------------


Another aspect of this issue.
Task failed on those 3 trackers because they have too little memory.
Hadoop scheduler correctly detected that fact, and correctly determined that each of those
trackers has the capacity of one slot.
Thus, the cluster has the total capacity of 53 closts (10 * 5 + 3 * 1).
However, the scheduler did not use that information when computing the maximum number of tasks
each tracker can take for the job, and thought each task tracker should take 4 tasks, regardless
of their actual capacity.

 

> Tasks are not scheduled even though task trackers have extra slots
> ------------------------------------------------------------------
>
>                 Key: HADOOP-5289
>                 URL: https://issues.apache.org/jira/browse/HADOOP-5289
>             Project: Hadoop Core
>          Issue Type: Bug
>          Components: mapred
>    Affects Versions: 0.19.0
>            Reporter: Runping Qi
>
> I ran a job with 51 reduce tasks on a cluster with 13 task trackers running Hadoop 0.19.
Each task tracker has 5 reduce slots.
> Initially, each task tracker accepted 4 reduce tasks as expected. However,  3 task trackers
were put into blacklist because many tasks failed on them.
> However, those failed tasks stayed in pending state, not being scheduled to other task
trackers, even though each of the other healthy tracker has one free slot.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message