hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Edward Capriolo (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HADOOP-5670) Hadoop configurations should be read from a distributed system
Date Wed, 12 May 2010 20:25:07 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-5670?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12866717#action_12866717

Edward Capriolo commented on HADOOP-5670:

I think we are thinking along the same lines. I and I understand why you did and it that way
but I wanted to chat about it.

commonname: string
hostname: multi-string
mapred.job.tracker: string
mapred.local.dir: multi-string
mapred.tasktracker.map.tasks.maximum: integer
mapred.tasktracker.reduce.tasks.maximum: integer

Doing this will cause you to have your LDAP schema generated from your hadoop configuration.
This is great because we can ensure type checking of the parameters but bad in a couple ways.

* we can not deal with non hadoop parameters (which should not be in the configuration but
could be)
* it will be a very intricate and large and we have to track new changes

Have your considered an alternate like this? 

commonname: string
hostname: multi-string
HadoopPropertyName: string
hadoopPropertyValue: multi-string

Yes, this is overly generic way is bad for several reasons, but I live it. Mostly because
right now the current configuration is an XML file. It has no XSD schema, hadoop does the
validity checking. What do you think? 

> Hadoop configurations should be read from a distributed system
> --------------------------------------------------------------
>                 Key: HADOOP-5670
>                 URL: https://issues.apache.org/jira/browse/HADOOP-5670
>             Project: Hadoop Common
>          Issue Type: New Feature
>          Components: conf
>            Reporter: Allen Wittenauer
> Rather than distributing the hadoop configuration files to every data node, compute node,
etc, Hadoop should be able to read configuration information (dynamically!) from LDAP, ZooKeeper,

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message