hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Steve Loughran (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HADOOP-6078) Ability to re-configure hadoop daemons online
Date Fri, 19 Jun 2009 13:15:07 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-6078?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12721776#action_12721776
] 

Steve Loughran commented on HADOOP-6078:
----------------------------------------

* You could do much of this just by taking down the specific nodes and bringing them up again.
* Hot reconfig is tricky, for as todd points out, some values are cached everywhere.
* Yet I'd like my TTs to be able to rebind to a JT that has just been brought up on a different
address.

At the very least, all the loops where workers spin waiting for their masters should reread
their configuration values every iteration. That way anything that has subclassed Configuration
to give live data could provide updated locations

FWIW, although I've subclassed  JobConf for my configuration, I stopped trying to do live
updates, as the objects end up being serialized and reread. I think it would be really hard
to do live reconfiguration with the current design. Node restart is cleaner. 

> Ability to re-configure hadoop daemons online
> ---------------------------------------------
>
>                 Key: HADOOP-6078
>                 URL: https://issues.apache.org/jira/browse/HADOOP-6078
>             Project: Hadoop Core
>          Issue Type: New Feature
>          Components: mapred
>            Reporter: Amar Kamat
>
> Example : 
> Like we have _bin hadoop mradmin -refreshNodes_ we should also have _bin hadoop mradmin
-reconfigure_ which re-configures mr while the cluster is online. Few parameters like job-expiry-interval
etc can be changed in this way without having to restart the whole cluster. 
> Master, once reconfigured, can ask the slaves to reconfigure (reload its config) from
a well defined location on hdfs or via heartbeat. 
> We can have some whitelisted configs that have _reloadable_ property. 

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message