hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "dhruba borthakur (JIRA)" <j...@apache.org>
Subject [jira] Updated: (MAPREDUCE-1361) In the pools with minimum slots, new job will always receive slots even if the minimum slots limit has been fulfilled
Date Mon, 12 Apr 2010 05:21:42 GMT

     [ https://issues.apache.org/jira/browse/MAPREDUCE-1361?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

dhruba borthakur updated MAPREDUCE-1361:
----------------------------------------

    Status: Open  (was: Patch Available)

> In the pools with minimum slots, new job will always receive slots even if the minimum
slots limit has been fulfilled
> ---------------------------------------------------------------------------------------------------------------------
>
>                 Key: MAPREDUCE-1361
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-1361
>             Project: Hadoop Map/Reduce
>          Issue Type: Improvement
>          Components: contrib/fair-share
>    Affects Versions: 0.20.2
>            Reporter: Scott Chen
>            Assignee: Scott Chen
>         Attachments: MAPREDUCE-1361-v1.patch
>
>
> In 0.20, the fair scheduler compares all the jobs based on their running tasks, minimum
slots and deficit. If the number of running tasks is less than the number of minimum slots,
it will be scheduled first.
> Consider a pool with minimum slot of 1000 but already have 5000 running tasks.
> If we launch another job on this pool, this new job will receive minimum slots based
on its weight. This new job may have higher weight if NewJobWeightBooster is used.
> So this new job will still get extra slots even if the pool's running tasks are way more
than the minimum slots.
> The latest version does not have this problem because it first compares pool then compares
jobs in the pool.

-- 
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: https://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message