hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hemanth Yamijala (JIRA)" <j...@apache.org>
Subject [jira] Commented: (MAPREDUCE-853) Support a hierarchy of queues in the Map/Reduce framework
Date Thu, 13 Aug 2009 11:14:14 GMT

    [ https://issues.apache.org/jira/browse/MAPREDUCE-853?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12742796#action_12742796
] 

Hemanth Yamijala commented on MAPREDUCE-853:
--------------------------------------------

The basic proposal is to define the concept of sub-queues. Sub-queues are queues that are
contained in other queues. Sub-queues can be nested. The last level of queues in the hierarchy,
the leaf level queues, are called job queues, as they contain jobs. By that definition, all
the queues that are defined in the present system are job queues.

An example organization could be:
{noformat}
grid {
  org1 {
    priority {
      production
      proj1
      proj2
      proj3
    }
    miscellaneous
  }
  org2
}
{noformat}

In this example, grid, org1, and priority are container queues. And production, proj1, proj2,
proj3, miscellaneous and org2 are job queues. An example of how policies such as capacity
can be assigned to this hierarchy and how it benefits them is described in MAPREDUCE-824.


> Support a hierarchy of queues in the Map/Reduce framework
> ---------------------------------------------------------
>
>                 Key: MAPREDUCE-853
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-853
>             Project: Hadoop Map/Reduce
>          Issue Type: New Feature
>          Components: jobtracker
>            Reporter: Hemanth Yamijala
>             Fix For: 0.21.0
>
>
> In MAPREDUCE-824, we proposed introducing a hierarchy of queues in the capacity scheduler.
Currently, the M/R framework provides the notion of job queues and handles some functionality
related to queues in a scheduler-agnostic manner. This functionality includes:
> - Managing the list of ACLs for queues
> - Managing the run state of queues - running or stopped
> - Displaying scheduling information about queues in the jobtracker web UI and job client
CLI
> - Displaying list of jobs in a queue in the jobtracker web UI and job client CLI
> - Providing APIs for list queues and queue information in JobClient.
> Since it would be beneficial to extend this functionality to hierarchical queues, this
JIRA is proposing introducing the concept into the map/reduce framework as well. We could
treat this as an umbrella JIRA and file additional tasks for each of the changes involved,
sticking to the high level approach in this JIRA.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message