hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Colin P. McCabe (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HADOOP-11223) Offer a read-only conf alternative to new Configuration()
Date Tue, 03 Jan 2017 16:12:58 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-11223?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15795391#comment-15795391
] 

Colin P. McCabe commented on HADOOP-11223:
------------------------------------------

If the real problem is reloading all those XML files all the time, why not change that behavior
in Hadoop 3.x?  At the very least, we could have some kind of mapping between classpath and
default configuration values, and only actually load the XML files when we saw a new classpath
which might cause us to load some different files.

> Offer a read-only conf alternative to new Configuration()
> ---------------------------------------------------------
>
>                 Key: HADOOP-11223
>                 URL: https://issues.apache.org/jira/browse/HADOOP-11223
>             Project: Hadoop Common
>          Issue Type: Bug
>          Components: conf
>            Reporter: Gopal V
>            Assignee: Varun Saxena
>              Labels: Performance
>         Attachments: HADOOP-11223.001.patch
>
>
> new Configuration() is called from several static blocks across Hadoop.
> This is incredibly inefficient, since each one of those involves primarily XML parsing
at a point where the JIT won't be triggered & interpreter mode is essentially forced on
the JVM.
> The alternate solution would be to offer a {{Configuration::getDefault()}} alternative
which disallows any modifications.
> At the very least, such a method would need to be called from 
> # org.apache.hadoop.io.nativeio.NativeIO::<clinit>()
> # org.apache.hadoop.security.SecurityUtil::<clinit>()
> # org.apache.hadoop.yarn.factory.providers.RecordFactoryProvider::<clinit>



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org


Mime
View raw message