Mailing-List: contact mapreduce-issues-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: mapreduce-issues@hadoop.apache.org
Message-ID: <580612491.1253181297549.JavaMail.jira@brutus>
Date: Thu, 17 Sep 2009 02:54:57 -0700 (PDT)
From: "Vinod K V (JIRA)" <jira@apache.org>
To: mapreduce-issues@hadoop.apache.org
Subject: [jira] Commented: (MAPREDUCE-893) Provide an ability to refresh
 queue configuration without restart.
In-Reply-To: <98754692.1250760074866.JavaMail.jira@brutus>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit


    [ https://issues.apache.org/jira/browse/MAPREDUCE-893?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12756446#action_12756446 ] 

Vinod K V commented on MAPREDUCE-893:
-------------------------------------


Also, for those who are listening, this is the new QueueRefresher interface that I am adding to TaskScheduler:

{code}
abstract class TaskScheduler implements Configurable {
   ......
   ......

  /**
   * Abstract QueueRefresher class. Scheduler's can extend this and return an
   * instance of this in the {@link #getQueueRefresher()} method. The
   * {@link #refreshQueues(List)} method of this instance will be invoked by the
   * {@link QueueManager} whenever it gets a request from an administrator to
   * refresh its own queue-configuration. This method has a documented contract
   * between the {@link QueueManager} and the {@link TaskScheduler}.
   */
  abstract class QueueRefresher {

    /**
     * Refresh the queue-configuration in the scheduler. This method has the
     * following contract.
     * <ol>
     * <li>Before this method, {@link QueueManager} does a validation of the new
     * queue-configuration. For e.g, currently addition of new queues, or
     * removal of queues at any level in the hierarchy is not supported by
     * {@link QueueManager} and so are not supported for schedulers too.</li>
     * <li>Schedulers will be passed a list of {@link JobQueueInfo}s of the root
     * queues i.e. the queues at the top level. All the descendants are properly
     * linked from these top-level queues.</li>
     * <li>Schedulers should use the scheduler specific queue properties from
     * the newRootQueues, validate the properties themselves and apply them
     * internally.</li>
     * <li>
     * Once the method returns successfully from the schedulers, it is assumed
     * that the refresh of queue properties is successful throughout and will be
     * 'committed' internally to {@link QueueManager} too. It is guaranteed that
     * at no point, after successful return from the scheduler, is the queue
     * refresh in QueueManager failed. If ever, such abnormalities happen, the
     * queue framework will be inconsistent and will need a JT restart.</li>
     * <li>If scheduler throws an exception during {@link #refreshQueues()},
     * {@link QueueManager} throws away the newly read configuration, retains
     * the old (consistent) configuration and informs the request issuer about
     * the error appropriately.</li>
     * </ol>
     * 
     * @param newRootQueues
     */
    abstract void refreshQueues(List<JobQueueInfo> newRootQueues)
        throws Throwable;
  }

  /**
   * Get the {@link QueueRefresher} for this scheduler. By default, no
   * {@link QueueRefresher} exists for a scheduler and is set to null.
   * Schedulers need to return an instance of {@link QueueRefresher} if they
   * wish to refresh their queue-configuration when {@link QueueManager}
   * refreshes its own queue-configuration via an administrator request.
   * 
   * @return
   */
  QueueRefresher getQueueRefresher() {
    return null;
  }
}
{code}

> Provide an ability to refresh queue configuration without restart.
> ------------------------------------------------------------------
>
>                 Key: MAPREDUCE-893
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-893
>             Project: Hadoop Map/Reduce
>          Issue Type: Improvement
>          Components: jobtracker
>            Reporter: Hemanth Yamijala
>            Assignee: Vinod K V
>             Fix For: 0.21.0
>
>         Attachments: MAPREDUCE-893-20090915.1.txt, MAPREDUCE-893-20090917.2.txt
>
>
> While administering a cluster using multiple queues, administrators feel a need to refresh queue properties on the fly without needing to restart the JobTracker. This is partially supported for some properties such as queue ACLs (HADOOP-5396) and state (HADOOP-5913). The idea is to extend the facility to refresh other queue properties as well, including scheduler properties.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.