hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Dave Latham (JIRA)" <j...@apache.org>
Subject [jira] [Resolved] (HBASE-4055) Client region location caches redundant HTableDescriptor's
Date Tue, 05 Jul 2011 16:59:16 GMT

     [ https://issues.apache.org/jira/browse/HBASE-4055?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Dave Latham resolved HBASE-4055.
--------------------------------

    Resolution: Duplicate

Ah, that should probably do it, Stack.  Thanks for the work.  Sorry for the duplicate issue.

It seems like too risky a change for the branch to me.  I look forward to seeing 0.92.

> Client region location caches redundant HTableDescriptor's
> ----------------------------------------------------------
>
>                 Key: HBASE-4055
>                 URL: https://issues.apache.org/jira/browse/HBASE-4055
>             Project: HBase
>          Issue Type: Improvement
>    Affects Versions: 0.90.3
>            Reporter: Dave Latham
>             Fix For: 0.92.0
>
>
> While examining the heap of a map task in a MapReduce job that writes directly to HBase,
I noticed that the HRegionLocation instances were taking up 90 MB (out of a 700 MB heap for
each map task) to cache the locations for 15K regions.  As the number of regions in the cluster
continues to grow, this continues to grow as well.
> Of that, it appears that about 80 MB were going to 15K HTableDescriptor instances.  There
are only 5 tables that it's writing to, so it seems to be wasting a great deal of memory with
a separate copy of the table descriptor for each region.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message