hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hemanth Yamijala (JIRA)" <j...@apache.org>
Subject [jira] Commented: (MAPREDUCE-861) Modify queue configuration format and parsing to support a hierarchy of queues.
Date Sat, 29 Aug 2009 08:39:32 GMT

    [ https://issues.apache.org/jira/browse/MAPREDUCE-861?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12749087#action_12749087

Hemanth Yamijala commented on MAPREDUCE-861:

I looked at most of the rest of the changes, including test cases. Here are some comments:

- Test case testXMLParsing() can cover queue state also.
- It should also verify that p1's child nodes are setup correctly.
- We need a test case to test default values in mapred-queues.xml, which will just have the
default queue, with no children and no properties, and acls disabled.
- More error conditions to check:
-- queue without a name
-- properties without key or value attributes
-- empty file without any queues.
- TestCapacitySchedulerConf tests must setup a map of queue names and properties and verify
that the conversion of properties to the appropriate data types (like int, float, boolean
etc). is happening correctly.

A few minor nits:

- CapacitySchedulerConf.addProperty can be called setProperties.
- CapacitySchedulerConf.getProperty should check for the queue first before getting the queue
properties. If we do this, we don't need to check for the queue not being null again, avoiding
a query.
- TestQueueManagerForHierarchialQueues.testXMLParsing needs correct formatting.

> Modify queue configuration format and parsing to support a hierarchy of queues.
> -------------------------------------------------------------------------------
>                 Key: MAPREDUCE-861
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-861
>             Project: Hadoop Map/Reduce
>          Issue Type: Sub-task
>            Reporter: Hemanth Yamijala
>            Assignee: rahul k singh
>         Attachments: MAPREDUCE-861-1.patch
> MAPREDUCE-853 proposes to introduce a hierarchy of queues into the Map/Reduce framework.
This JIRA is for defining changes to the configuration related to queues. 
> The current format for defining a queue and its properties is as follows: mapred.queue.<queue-name>.<property-name>.
For e.g. mapred.queue.<queue-name>.acl-submit-job. The reason for using this verbose
format was to be able to reuse the Configuration parser in Hadoop. However, administrators
currently using the queue configuration have already indicated a very strong desire for a
more manageable format. Since, this becomes more unwieldy with hierarchical queues, the time
may be good to introduce a new format for representing queue configuration.

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message