hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Yi Liu (JIRA)" <j...@apache.org>
Subject [jira] [Resolved] (HDFS-8912) Implement ShrinkableHashMap extends java HashMap and use properly
Date Tue, 18 Aug 2015 13:21:46 GMT

     [ https://issues.apache.org/jira/browse/HDFS-8912?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel

Yi Liu resolved HDFS-8912.
    Resolution: Invalid

I forgot one thing: many variables/methods of java {{HashMap}} we need to touch in {{ShrinkableHashMap}}
are {{package-private}}, so we can't simply extend {{HashMap}}. Implementing a new {{ShrinkableHashMap}}
is a bit heavy, close it as "Invalid".

> Implement ShrinkableHashMap extends java HashMap and use properly
> -----------------------------------------------------------------
>                 Key: HDFS-8912
>                 URL: https://issues.apache.org/jira/browse/HDFS-8912
>             Project: Hadoop HDFS
>          Issue Type: Sub-task
>            Reporter: Yi Liu
>            Assignee: Yi Liu
> Currently {{LightWeightHashSet}} and {{LightWeightLinkedSet}} are used in hdfs, there
are two advantages compared to java HashSet: one is the entry requires fewer memory, another
is it's shrinkable.  In real cluster, hdfs is a long running service, and {{set}} may become
large at some time and may become small after that, so shrinking the {{set}} when size hits
the shrink threshold is necessary, it can improve the NN memory.
> Same situation for {{map}}, some HashMap used in BlockManager (e.g., the hashmap in CorruptReplicasMap),
it's better to be shrinkable. 
>  I think it's worth to implement ShrinkableHashMap extends the java HashMap, for quick
glance, seems few code is needed.

This message was sent by Atlassian JIRA

View raw message