hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Andrew Wang (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HADOOP-11506) Configuration.get() is unnecessarily slow
Date Thu, 22 Jan 2015 23:35:34 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-11506?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14288454#comment-14288454
] 

Andrew Wang commented on HADOOP-11506:
--------------------------------------

Patch here would be most welcome, especially if you're willing to fire up {{perf}} and get
some pretty numbers :)

I flamegraph'd some MR jobs about a year ago and noticed a lot of time spent in Configuration,
so I'd love to see this improved.

> Configuration.get() is unnecessarily slow
> -----------------------------------------
>
>                 Key: HADOOP-11506
>                 URL: https://issues.apache.org/jira/browse/HADOOP-11506
>             Project: Hadoop Common
>          Issue Type: Bug
>            Reporter: Dmitriy V. Ryaboy
>
> Profiling several large Hadoop jobs, we discovered that a surprising amount of time was
spent inside Configuration.get, more specifically, in regex matching caused by the substituteVars
call.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message