hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hudson (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HADOOP-15554) Improve JIT performance for Configuration parsing
Date Mon, 02 Jul 2018 16:37:00 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-15554?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16530158#comment-16530158

Hudson commented on HADOOP-15554:

FAILURE: Integrated in Jenkins build Hadoop-trunk-Commit #14509 (See [https://builds.apache.org/job/Hadoop-trunk-Commit/14509/])
HADOOP-15554. Improve JIT performance for Configuration parsing. (wang: rev f51da9c4d1423c2ac92eb4f40e973264e7e968cc)
* (edit) hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/conf/Configuration.java

> Improve JIT performance for Configuration parsing
> -------------------------------------------------
>                 Key: HADOOP-15554
>                 URL: https://issues.apache.org/jira/browse/HADOOP-15554
>             Project: Hadoop Common
>          Issue Type: Improvement
>          Components: conf, performance
>    Affects Versions: 3.0.0
>            Reporter: Todd Lipcon
>            Assignee: Todd Lipcon
>            Priority: Minor
>             Fix For: 3.2.0, 3.1.1, 3.0.4
>         Attachments: HADOOP-15554.branch-3.0.patch, HADOOP-15554.branch-3.0.patch, hadoop-15554.patch,
> In investigating a performance regression for small tasks between Hadoop 2 and Hadoop
3, we found that the amount of time spent in JIT was significantly higher. Using jitwatch
we were able to determine that, due to a combination of switching from DOM to SAX style parsing
and just having more configuration key/value pairs, Configuration.loadResource is now getting
compiled with the C2 compiler and taking quite some time. Breaking that very large function
up into several smaller ones and eliminating some redundant bits of code improves the JIT
performance measurably.

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org

View raw message