hadoop-hdfs-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Colin Patrick McCabe (JIRA)" <j...@apache.org>
Subject [jira] [Created] (HDFS-6709) Implement off-heap data structures for NameNode and other HDFS memory optimization
Date Fri, 18 Jul 2014 23:54:40 GMT
Colin Patrick McCabe created HDFS-6709:
------------------------------------------

             Summary: Implement off-heap data structures for NameNode and other HDFS memory
optimization
                 Key: HDFS-6709
                 URL: https://issues.apache.org/jira/browse/HDFS-6709
             Project: Hadoop HDFS
          Issue Type: Improvement
          Components: namenode
            Reporter: Colin Patrick McCabe
            Assignee: Colin Patrick McCabe


We should investigate implementing off-heap data structures for NameNode and other HDFS memory
optimization.  These data structures could reduce latency by avoiding the long GC times that
occur with large Java heaps.  We could also avoid per-object memory overheads and control
memory layout a little bit better.  This also would allow us to use the JVM's "compressed
oops" optimization even with really large namespaces, if we could get the Java heap below
32 GB for those cases.  This would provide another performance and memory efficiency boost.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

Mime
View raw message