hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Vivek Ratan (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HADOOP-4522) Capacity Scheduler needs to re-read its configuration
Date Fri, 14 Nov 2008 03:52:44 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-4522?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12647512#action_12647512

Vivek Ratan commented on HADOOP-4522:

Adding to Dhruba's comments. 

I suppose it's up to the component to decide what to re-read and what not to? For example,
do we want the QueueManager to re-read the list of all queues in the system so that it can
dynamically add new queues or delete existing queues? probably not, as it will affect jobs
in current queues (you can't 'delete' a queue if it has jobs running or waiting). No matter
what we do, components that depend on the QueueManager need to know that. The CapacityScheduler,
for example, needs to know whether new queues are accepted, so that it may, in turn, read
(or not read) scheduling information related to the new queues. 

I think that having the system re-read its configuration through a single call is OK, but
we may need to make it explicit, in documentation, what config values are re-read and what
are ignored. We'd also want admins to be explicitly aware that ALL config files are re-read.
OTOH, you way want to break down config information into sub-sections (config for scheduling,
for JT, etc), and provide one or more sub-sections as parameters in the call, but this might
get messy. 

Beyond scheduling, has there been a need to have the system re-read its configuration? 

> Capacity Scheduler needs to re-read its configuration
> -----------------------------------------------------
>                 Key: HADOOP-4522
>                 URL: https://issues.apache.org/jira/browse/HADOOP-4522
>             Project: Hadoop Core
>          Issue Type: New Feature
>            Reporter: Vivek Ratan
> An external application (an Ops script, or some CLI-based tool) can change the configuration
of the Capacity Scheduler (change the capacities of various queues, for example) by updating
its config file. This application then needs to tell the Capacity Scheduler that its config
has changed, which causes the Scheduler to re-read its configuration. It's possible that the
Capacity Scheduler may need to interact with external applications in other similar ways.

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message