hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "rahul k singh (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HADOOP-5913) Allow administrators to be able to start and stop queues
Date Wed, 27 May 2009 04:54:45 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-5913?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12713413#action_12713413
] 

rahul k singh commented on HADOOP-5913:
---------------------------------------

Summary:
  -This feature would provide functionality to stop and start queues in Hadoop at runtime.
All the running jobs in the queue will be 
   completed , and any further new jobs will not be accepted. This state would be persisted
in the configuration file.	

Requirements:
 - Administrators should be able to stop(stop accepting jobs) and start(start accepting jobs)

   queues at runtime.
 - if a queue is stopped at runtime , it should complete all the existing running job and
stop
   accepting any new jobs.
 - Administators should be able to change the queue configuration to start and stop.
 - Once configuration is updated , administrators run " hadoop refreshQueues" command to refresh
the existing
   queues.This configuration would be persisted across the Jobtracker restart.
 - Moving all the queue info(ACLS and state(stop or running)) in a common xml file , this
would reduce the no of 
   commands required to change any queue settings , we can simply change this xml file and
execute "hadoop refreshQueues"
   to update the queue information. 
 - In order to do the above , we would be renaming mapred-queue-acls.xml file to mapred-queues.xml.

Design

-Assumptions:
  - Administrators would have permission to execute the "hadoop refreshQueues"  command.
  - Only settings which are mentioned in mapred-queue.xml would be changed , any queue settings
which are not part of 
   mapred-queue.xml(For ex. settings in scheduler's configuration) will not be effected. 
-Summary
  - Rename the"mapred-queue-acls.xml" to "mapred-queues.xml"
  - Move all the queue related data from mapred-site.xml to mapred-queues.xml
    Following tags would be moved.
    - mapred.queue.names
    - mapred.acls.enabled
    - mapred.queue.<queueName>.acl-submit-job //This setting is for each queue
    - mapred.queue.<queueName>.acl-administer-job //This setting is for each queue
  - Refactor the existing code to encapsulate this new mapred-queues.xml file
  - Introduce the new command "hadoop refreshQueues" which reads from the existing mapred-queues.xml
  - Introduce new api in QueuesManager.java to check for state of the Queue.
  - Introduction of new property for Queue in mapred-queues.xml.
    "mapred.queue.<queueName>.state" values being "stopped" | "running"


> Allow administrators to be able to start and stop queues 
> ---------------------------------------------------------
>
>                 Key: HADOOP-5913
>                 URL: https://issues.apache.org/jira/browse/HADOOP-5913
>             Project: Hadoop Core
>          Issue Type: New Feature
>          Components: mapred
>            Reporter: rahul k singh
>
>   This feature would provide functionality to stop and start queues in Hadoop at runtime.


-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message