hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Todd Lipcon (JIRA)" <j...@apache.org>
Subject [jira] [Created] (HADOOP-15554) Improve JIT performance for Configuration parsing
Date Thu, 21 Jun 2018 17:51:00 GMT
Todd Lipcon created HADOOP-15554:

             Summary: Improve JIT performance for Configuration parsing
                 Key: HADOOP-15554
                 URL: https://issues.apache.org/jira/browse/HADOOP-15554
             Project: Hadoop Common
          Issue Type: Improvement
          Components: conf, performance
    Affects Versions: 3.0.0
            Reporter: Todd Lipcon
            Assignee: Todd Lipcon

In investigating a performance regression for small tasks between Hadoop 2 and Hadoop 3, we
found that the amount of time spent in JIT was significantly higher. Using jitwatch we were
able to determine that, due to a combination of switching from DOM to SAX style parsing and
just having more configuration key/value pairs, Configuration.loadResource is now getting
compiled with the C2 compiler and taking quite some time. Breaking that very large function
up into several smaller ones and eliminating some redundant bits of code improves the JIT
performance measurably.

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org

View raw message