hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Yi Liu (JIRA)" <j...@apache.org>
Subject [jira] [Created] (HDFS-8862) Improve BlockManager#excessReplicateMap
Date Thu, 06 Aug 2015 01:32:04 GMT
Yi Liu created HDFS-8862:

             Summary: Improve BlockManager#excessReplicateMap
                 Key: HDFS-8862
                 URL: https://issues.apache.org/jira/browse/HDFS-8862
             Project: Hadoop HDFS
          Issue Type: Sub-task
          Components: namenode
            Reporter: Yi Liu
            Assignee: Yi Liu

Per [~cmccabe]'s comments in HDFS-8792, this JIRA is to discuss improving {{BlockManager#excessReplicateMap}}.

That's right HashMap don't ever shrink when elements are removed,  but TreeMap entry needs
to store more (memory) references (left,  right, parent) than HashMap entry (only one reference
next),  even when there is element removing and cause some entry empty, the empty HashMap
entry is just a {{null}} reference (4 bytes),  so they are close at this point.  On the other
hand, the key of {{excessReplicateMap}} is datanode uuid, so the entries number is almost
fixed, so HashMap memory is good than TreeMap memory in this case.   I think the most important
is the search/insert/remove performance, HashMap is absolutely better than TreeMap.  Because
we don't need to sort,  we should use HashMap instead of TreeMap

This message was sent by Atlassian JIRA

View raw message