hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Vinod K V (JIRA)" <j...@apache.org>
Subject [jira] Commented: (MAPREDUCE-893) Provide an ability to refresh queue configuration without restart.
Date Thu, 17 Sep 2009 09:54:57 GMT

    [ https://issues.apache.org/jira/browse/MAPREDUCE-893?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12756446#action_12756446

Vinod K V commented on MAPREDUCE-893:

Also, for those who are listening, this is the new QueueRefresher interface that I am adding
to TaskScheduler:

abstract class TaskScheduler implements Configurable {

   * Abstract QueueRefresher class. Scheduler's can extend this and return an
   * instance of this in the {@link #getQueueRefresher()} method. The
   * {@link #refreshQueues(List)} method of this instance will be invoked by the
   * {@link QueueManager} whenever it gets a request from an administrator to
   * refresh its own queue-configuration. This method has a documented contract
   * between the {@link QueueManager} and the {@link TaskScheduler}.
  abstract class QueueRefresher {

     * Refresh the queue-configuration in the scheduler. This method has the
     * following contract.
     * <ol>
     * <li>Before this method, {@link QueueManager} does a validation of the new
     * queue-configuration. For e.g, currently addition of new queues, or
     * removal of queues at any level in the hierarchy is not supported by
     * {@link QueueManager} and so are not supported for schedulers too.</li>
     * <li>Schedulers will be passed a list of {@link JobQueueInfo}s of the root
     * queues i.e. the queues at the top level. All the descendants are properly
     * linked from these top-level queues.</li>
     * <li>Schedulers should use the scheduler specific queue properties from
     * the newRootQueues, validate the properties themselves and apply them
     * internally.</li>
     * <li>
     * Once the method returns successfully from the schedulers, it is assumed
     * that the refresh of queue properties is successful throughout and will be
     * 'committed' internally to {@link QueueManager} too. It is guaranteed that
     * at no point, after successful return from the scheduler, is the queue
     * refresh in QueueManager failed. If ever, such abnormalities happen, the
     * queue framework will be inconsistent and will need a JT restart.</li>
     * <li>If scheduler throws an exception during {@link #refreshQueues()},
     * {@link QueueManager} throws away the newly read configuration, retains
     * the old (consistent) configuration and informs the request issuer about
     * the error appropriately.</li>
     * </ol>
     * @param newRootQueues
    abstract void refreshQueues(List<JobQueueInfo> newRootQueues)
        throws Throwable;

   * Get the {@link QueueRefresher} for this scheduler. By default, no
   * {@link QueueRefresher} exists for a scheduler and is set to null.
   * Schedulers need to return an instance of {@link QueueRefresher} if they
   * wish to refresh their queue-configuration when {@link QueueManager}
   * refreshes its own queue-configuration via an administrator request.
   * @return
  QueueRefresher getQueueRefresher() {
    return null;

> Provide an ability to refresh queue configuration without restart.
> ------------------------------------------------------------------
>                 Key: MAPREDUCE-893
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-893
>             Project: Hadoop Map/Reduce
>          Issue Type: Improvement
>          Components: jobtracker
>            Reporter: Hemanth Yamijala
>            Assignee: Vinod K V
>             Fix For: 0.21.0
>         Attachments: MAPREDUCE-893-20090915.1.txt, MAPREDUCE-893-20090917.2.txt
> While administering a cluster using multiple queues, administrators feel a need to refresh
queue properties on the fly without needing to restart the JobTracker. This is partially supported
for some properties such as queue ACLs (HADOOP-5396) and state (HADOOP-5913). The idea is
to extend the facility to refresh other queue properties as well, including scheduler properties.

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message