hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Gopal V (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HADOOP-9570) Configuration.addResource() should only parse the new resource
Date Tue, 21 May 2013 17:27:16 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-9570?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13663154#comment-13663154

Gopal V commented on HADOOP-9570:

Patch still fails some tests, shelving for future investigation.
> Configuration.addResource() should only parse the new resource
> --------------------------------------------------------------
>                 Key: HADOOP-9570
>                 URL: https://issues.apache.org/jira/browse/HADOOP-9570
>             Project: Hadoop Common
>          Issue Type: Improvement
>          Components: conf
>         Environment: Ubuntu LXC
>            Reporter: Gopal V
>            Assignee: Gopal V
>            Priority: Minor
>         Attachments: HADOOP-9750.patch
> Hadoop configuration parsing re-parses all configuration files when a new resource is
added to the configuration object.
> This is wasteful and runs through every deprecation check unnecessarily.
> {code}
> JobConf clone = new JobConf(job);
> clone.addResource(...);
> {code}
> will reparse every file including core-site, hdfs-site etc.
> Resource addition can be taken care of cleanly, if the addResourceObject() call applies
the new resource, followed by applying the overlay, without re-parsing any of the older files
already loaded into the hashtable.

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

View raw message