hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Vinod K V (JIRA)" <j...@apache.org>
Subject [jira] Commented: (MAPREDUCE-1723) Capacity Scheduler should allow configuration of Map & Reduce task slots independently per queue
Date Mon, 26 Apr 2010 08:31:36 GMT

    [ https://issues.apache.org/jira/browse/MAPREDUCE-1723?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12860860#action_12860860
] 

Vinod K V commented on MAPREDUCE-1723:
--------------------------------------

This jira-ask seems to be more aligned to the enhancement proposal in point (1) than preemption.
From my understanding, this issue is about, to give some examples, queues that have only-map
jobs or jobs which always need significantly higher percentage of maps than reduces. In that
case, even if we have preemption, we still may want to configure the queue to have lower share
of reduces w.r.t other queues independent of the share of maps, no?

> Capacity Scheduler should allow configuration of Map & Reduce task slots independently
per queue
> ------------------------------------------------------------------------------------------------
>
>                 Key: MAPREDUCE-1723
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-1723
>             Project: Hadoop Map/Reduce
>          Issue Type: Improvement
>          Components: contrib/capacity-sched
>    Affects Versions: 0.20.1
>         Environment: all
>            Reporter: Subramaniam Krishnan
>             Fix For: 0.20.3
>
>
> The Capacity Scheduler allows configuration of percentage of task slots per queue. We
have a scenario in which our biggest queue (50% quota) has Jobs with mainly Map tasks &
we need to enforce strict capacity limits per queue due to SLA requirements. So other smaller
queues which require Reduce tasks gets starved even though the Reduce slots are idle. The
Grid can be more efficiently utilized if Capacity Scheduler allows configuration of Map &
Reduce task slots capacity independently per queue.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message