hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Ari Rabkin (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HADOOP-7036) spellcheck for configuration
Date Mon, 29 Nov 2010 20:17:15 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-7036?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12964916#action_12964916
] 

Ari Rabkin commented on HADOOP-7036:
------------------------------------

The jar is now in Maven. I'll revise the patch to pull it in that way.

The tool is intended for ops folks to some extent, and novice users to an even greater extent.
I'm not sure which way that pushes the packaging question. As I understand, this is different
in the v20 branch and the v21 branch. Is there a document somewhere summarizing what goes
in contrib for each Hadoop branch?

I don't understand the right way to divide stuff up amongst projects. Seems like a hassle
to have the Mapred dictionary in one patch against one project, the HDFS dictionary in another,
and the common options and the invoke script in a third. Can that really be the right way
to go?



> spellcheck for configuration
> ----------------------------
>
>                 Key: HADOOP-7036
>                 URL: https://issues.apache.org/jira/browse/HADOOP-7036
>             Project: Hadoop Common
>          Issue Type: New Feature
>          Components: conf
>            Reporter: Ari Rabkin
>            Assignee: Ari Rabkin
>         Attachments: confspellcheck.jar, hadoopSpellcheck.patch
>
>
> Hadoop does fairly limited correctness checks of its configuration. I propose a "configuration
spellcheck" that can automatically catch errors, and particularly can catch cases where users
mis-type the name of an option.
> The system works as follows:
> - Use program analysis to extract the set of options supported by each Hadoop version,
annotated when possible with their types into a 'dictionary file'.
> - Distribute these extracted sets, per version.
> - A script that reads a dictionary file, reads the Hadoop config from a specified directory,
and reports deviations. In particular, the system can report when an option is set that Hadoop
will never read or when an invalid value is specified.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message