hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Colin Patrick McCabe (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-7836) BlockManager Scalability Improvements
Date Fri, 06 Mar 2015 20:54:41 GMT

    [ https://issues.apache.org/jira/browse/HDFS-7836?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14350878#comment-14350878
] 

Colin Patrick McCabe commented on HDFS-7836:
--------------------------------------------

I don't think the block map size is really that easy to get at right now.  Taking a heap dump
on a big NN can take minutes... it's not something most sysadmins will let you do.  And the
analysis is difficult... a lot of common heap analysis tools require tons of memory.

Anyway, we should probably add a JMX counter for the size(s) of the block map hash tables,
and the number of entries, for tracking purposes.

> BlockManager Scalability Improvements
> -------------------------------------
>
>                 Key: HDFS-7836
>                 URL: https://issues.apache.org/jira/browse/HDFS-7836
>             Project: Hadoop HDFS
>          Issue Type: Improvement
>            Reporter: Charles Lamb
>            Assignee: Charles Lamb
>         Attachments: BlockManagerScalabilityImprovementsDesign.pdf
>
>
> Improvements to BlockManager scalability.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message