hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Konstantin Boudnik (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HADOOP-7036) spellcheck for configuration
Date Wed, 17 Nov 2010 20:33:15 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-7036?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12933141#action_12933141
] 

Konstantin Boudnik commented on HADOOP-7036:
--------------------------------------------

Good idea, I like it. However the jar file needs to be included via Ivy dependency declaration
- not directly to the SVN.

Also, it seems that it needs to be split between the project. E.g. Common shouldn't know anything
about HDFS or MR specific configuration options.

One more nit: the tool sounds more like a nice addition to Ops (cluster operation) folks or
whoever else needs to create their own configurations. Perhaps it belongs to HDFS/MR contribs
rathen than Common.

+1 on the idea, though!

> spellcheck for configuration
> ----------------------------
>
>                 Key: HADOOP-7036
>                 URL: https://issues.apache.org/jira/browse/HADOOP-7036
>             Project: Hadoop Common
>          Issue Type: New Feature
>          Components: conf
>            Reporter: Ari Rabkin
>            Assignee: Ari Rabkin
>         Attachments: confspellcheck.jar, hadoopSpellcheck.patch
>
>
> Hadoop does fairly limited correctness checks of its configuration. I propose a "configuration
spellcheck" that can automatically catch errors, and particularly can catch cases where users
mis-type the name of an option.
> The system works as follows:
> - Use program analysis to extract the set of options supported by each Hadoop version,
annotated when possible with their types into a 'dictionary file'.
> - Distribute these extracted sets, per version.
> - A script that reads a dictionary file, reads the Hadoop config from a specified directory,
and reports deviations. In particular, the system can report when an option is set that Hadoop
will never read or when an invalid value is specified.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message