hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Devaraj Das (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HADOOP-3651) When assigning tasks to trackers, the job tracker should try to balance the number of tasks among the available trackers
Date Fri, 27 Jun 2008 12:33:45 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-3651?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12608751#action_12608751
] 

Devaraj Das commented on HADOOP-3651:
-------------------------------------

Runping, can you tell us whether there was more than one job simulataneously active in the
cluster?
I don't think that the load factor calculation is affecting this. If you take a cluster with
200 tasktrackers and a job with 200 reducers (and that is the only job in the cluster), then
it is guaranteed (as per the load factor code) that no more than one reduce task will be assigned
to a tasktracker. Of course, if you have things like lost trackers then some tasktrackers
will run more than one.

> When assigning tasks to trackers, the job tracker should try to balance the number of
tasks among the available trackers
> ------------------------------------------------------------------------------------------------------------------------
>
>                 Key: HADOOP-3651
>                 URL: https://issues.apache.org/jira/browse/HADOOP-3651
>             Project: Hadoop Core
>          Issue Type: Improvement
>          Components: mapred
>    Affects Versions: 0.17.0
>            Reporter: Runping Qi
>
> I encounter a number of situations like this:
> A job tracker has 200 task trackers, each with 2 mapper slots and reducer slots.
> When a job with 200 or fewer reducers was submitted to the job tracker,
> one normally each task tracker will run one reducer.
> Unfortunately, it seems that only  about 1/3 of trackers have one reducer, and 1/3 trackers
don't have reducer, and 1/3 have 2 reducers!

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message