hadoop-hdfs-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Yi Liu (JIRA)" <j...@apache.org>
Subject [jira] [Created] (HDFS-8912) Implement ShrinkableHashMap extends java HashMap and use properly
Date Tue, 18 Aug 2015 01:57:46 GMT
Yi Liu created HDFS-8912:
----------------------------

             Summary: Implement ShrinkableHashMap extends java HashMap and use properly
                 Key: HDFS-8912
                 URL: https://issues.apache.org/jira/browse/HDFS-8912
             Project: Hadoop HDFS
          Issue Type: Sub-task
            Reporter: Yi Liu
            Assignee: Yi Liu


Currently {{LightWeightHashSet}} and {{LightWeightLinkedSet}} are used in hdfs, there are
two advantages compared to java HashSet: one is the entry requires fewer memory, another is
it's shrinkable.  In real cluster, hdfs is a long running service, and {{set}} may become
very large at some time and may become small after that, so shrinking the {{set}} when size
hits the shrink threshold is necessary, it can improve the NN memory.

Same situation for {{map}}, some HashMap used in BlockManager (e.g., the hashmap in CorruptReplicasMap),
it's better to be shrinkable. 
 I think it's worth to implement ShrinkableHashMap extends the java HashMap, for quick glance,
seems few code is needed.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message