hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Subramaniam Krishnan (JIRA)" <j...@apache.org>
Subject [jira] Commented: (MAPREDUCE-1723) Capacity Scheduler should allow configuration of Map & Reduce task slots independently per queue
Date Fri, 23 Apr 2010 09:37:51 GMT

    [ https://issues.apache.org/jira/browse/MAPREDUCE-1723?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12860195#action_12860195
] 

Subramaniam Krishnan commented on MAPREDUCE-1723:
-------------------------------------------------


Hemanth, the proportion of Map & Reduce slots in our cluster would ideally be 3:4 (tending
to even 4:5) but right now we have configured it at 2:3 to mitigate the effect described above.

> Capacity Scheduler should allow configuration of Map & Reduce task slots independently
per queue
> ------------------------------------------------------------------------------------------------
>
>                 Key: MAPREDUCE-1723
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-1723
>             Project: Hadoop Map/Reduce
>          Issue Type: Improvement
>          Components: contrib/capacity-sched
>    Affects Versions: 0.20.1
>         Environment: all
>            Reporter: Subramaniam Krishnan
>             Fix For: 0.20.3
>
>
> The Capacity Scheduler allows configuration of percentage of task slots per queue. We
have a scenario in which our biggest queue (50% quota) has Jobs with mainly Map tasks &
we need to enforce strict capacity limits per queue due to SLA requirements. So other smaller
queues which require Reduce tasks gets starved even though the Reduce slots are idle. The
Grid can be more efficiently utilized if Capacity Scheduler allows configuration of Map &
Reduce task slots capacity independently per queue.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message