Mailing-List: contact issues-help@hbase.apache.org; run by ezmlm
Precedence: bulk
Date: Fri, 1 Jul 2011 23:51:28 +0000 (UTC)
From: "Dave Latham (JIRA)" <jira@apache.org>
To: issues@hbase.apache.org
Message-ID: 
 <1362990750.10168.1309564288363.JavaMail.tomcat@hel.zones.apache.org>
Subject: [jira] [Created] (HBASE-4055) Client region location caches
 redundant HTableDescriptor's
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit

Client region location caches redundant HTableDescriptor's
----------------------------------------------------------

                 Key: HBASE-4055
                 URL: https://issues.apache.org/jira/browse/HBASE-4055
             Project: HBase
          Issue Type: Improvement
    Affects Versions: 0.90.3
            Reporter: Dave Latham
             Fix For: 0.92.0


While examining the heap of a map task in a MapReduce job that writes directly to HBase, I noticed that the HRegionLocation instances were taking up 90 MB (out of a 700 MB heap for each map task) to cache the locations for 15K regions.  As the number of regions in the cluster continues to grow, this continues to grow as well.

Of that, it appears that about 80 MB were going to 15K HTableDescriptor instances.  There are only 5 tables that it's writing to, so it seems to be wasting a great deal of memory with a separate copy of the table descriptor for each region.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira