hadoop-mapreduce-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Dheeren Beborrtha (JIRA)" <j...@apache.org>
Subject [jira] [Created] (MAPREDUCE-4265) Support Queue type exclusive or shared
Date Wed, 16 May 2012 23:53:06 GMT
Dheeren Beborrtha created MAPREDUCE-4265:

             Summary: Support Queue type exclusive or shared
                 Key: MAPREDUCE-4265
                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-4265
             Project: Hadoop Map/Reduce
          Issue Type: Improvement
          Components: contrib/capacity-sched
    Affects Versions:
            Reporter: Dheeren Beborrtha

In the current implementation Capacity, MaxCapacity are the two main parameters that can be
used for sharing resources on the cluster.  But there is no feature to protect the resource
of a queue. In the absence of pre-emption as a feature, if someone decides to reserve its
capacity for its exclusive use,  while rest of the queues are set for sharing, it is not simply
If we choose one queue to be set for exclusive use by setting Capacity=MaxCapacity, this queue
gets penalized, if others choose to set for sharing with their MaxCapacity more than capacity.
This is an unintended consequence. On the other hand ,  if every queue is  set to  its Capacity=MaxCapacity
 the cluster is being partitioned into distinct virtual clusters.
So, the suggestion is, to add an additional queue parameter  <queue_type> with value
<exclusive> or <shared>. By default a queue is always shared type. If queue_type
is exclusive, the capacity of this queue is non sharable and should be taken out from  total
capacity while scheduling jobs in other queues.

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira


View raw message