hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Owen O'Malley (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HADOOP-2866) JobConf should validate key names in well-defined namespaces and warn on misspelling
Date Thu, 21 Feb 2008 17:29:19 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-2866?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12571107#action_12571107
] 

Owen O'Malley commented on HADOOP-2866:
---------------------------------------

For every variable there should be an API that sets and gets the variable. That is much more
stable than having a registry and they can be tracked by the API scanning tools.

And yes, the naming of the variables is adhoc and should (in theory!) be cleaned up. The problem
with cleaning them up is exactly the people that don't  always use the APIs for setting and
getting the values...

> JobConf should validate key names in well-defined namespaces and warn on misspelling
> ------------------------------------------------------------------------------------
>
>                 Key: HADOOP-2866
>                 URL: https://issues.apache.org/jira/browse/HADOOP-2866
>             Project: Hadoop Core
>          Issue Type: Improvement
>          Components: mapred
>    Affects Versions: 0.16.0
>            Reporter: Aaron Kimball
>            Priority: Minor
>             Fix For: 0.16.1, 0.17.0
>
>   Original Estimate: 72h
>  Remaining Estimate: 72h
>
> A discussion on the mailing list reveals that some configuration strings in the JobConf
are deprecated over time and new configuration names replace them:
> e.g., "mapred.output.compression.type" is now replaced with "mapred.map.output.compression.type"
> Programmers who have been manually specifying the former string, however, receive no
diagnostic output during testing to suggest that their compression type is being silently
ignored.
> It would be desirable to notify developers of this change by printing a warning message
when deprecated configuration names are used in a newer version of Hadoop. More generally,
when any configuration string in the mapred.\*, fs.\*, dfs.\*, etc namespaces are provided
by a user and are not recognized by Hadoop, it is desirable to print a warning, to indicate
malformed configurations. No warnings should be printed when configuration keys are in user-defined
namespaces (e.g., "myprogram.mytask.myvalue").

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message