hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "He Tianyi (JIRA)" <j...@apache.org>
Subject [jira] [Created] (HADOOP-13390) GC pressure of NetworkTopology.isAncestor call can be eliminated
Date Wed, 20 Jul 2016 05:15:21 GMT
He Tianyi created HADOOP-13390:

             Summary: GC pressure of NetworkTopology.isAncestor call can be eliminated
                 Key: HADOOP-13390
                 URL: https://issues.apache.org/jira/browse/HADOOP-13390
             Project: Hadoop Common
          Issue Type: Improvement
          Components: net
    Affects Versions: 2.7.0, 2.6.0, 2.8.0
            Reporter: He Tianyi
            Priority: Minor

{{NetworkTopology.isAncestor}} is called in {{NetworkTopology.getLeaf}}, which is excessively
used in block placement policies ({{chooseRandom}} for example).

Currently, the implementation calls {{getPath}} twice. And {{getPath}} performs string concatenation
on the fly. On a busy NameNode, this introduces more GC pressure and CPU overhead for block
Given that network location and node name does not generally change frequently, we can cache
path as a properly of {{Node}} and update accordingly while network location or node name
is changing.
Also, one of these {{getPath}} calls in {{getLeaf}} can be eliminated because we are expecting
identical result in both calls.

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org

View raw message