Mailing-List: contact hbase-dev-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: hbase-dev@hadoop.apache.org
Message-ID: <1972376326.1228946084334.JavaMail.jira@brutus>
Date: Wed, 10 Dec 2008 13:54:44 -0800 (PST)
From: "stack (JIRA)" <jira@apache.org>
To: hbase-dev@hadoop.apache.org
Subject: [jira] Commented: (HBASE-900) Regionserver memory leak causing OOME
 during relatively modest bulk importing
In-Reply-To: <545424030.1222300426159.JavaMail.jira@brutus>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit


    [ https://issues.apache.org/jira/browse/HBASE-900?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12655378#action_12655378 ] 

stack commented on HBASE-900:
-----------------------------

Our calculation of MemCache sizes is way off.  Our math says the aggregate of all Memcaches is 200MB.  In the profiler, the 153 Memcaches present on OOME have 800MBs accumulated.  Working on a better memcache sizer.

Other items, the presence of compressors/decompressors is 'normal'.  A mapfile index is block compressed.  Bad news is that though the index file is closed as soon as possible, allocated buffers for decompressors stick around (MapFile keeps reference to the index SequenceFile so its not GC'd).  OK news is that in scheme of things, accounts for small amount of heap -- about 10MB in tim's case.

> Regionserver memory leak causing OOME during relatively modest bulk importing
> -----------------------------------------------------------------------------
>
>                 Key: HBASE-900
>                 URL: https://issues.apache.org/jira/browse/HBASE-900
>             Project: Hadoop HBase
>          Issue Type: Bug
>    Affects Versions: 0.18.1, 0.19.0
>            Reporter: Jonathan Gray
>            Assignee: stack
>            Priority: Blocker
>         Attachments: 900.patch, memoryOn13.png
>
>
> I have recreated this issue several times and it appears to have been introduced in 0.2.
> During an import to a single table, memory usage of individual region servers grows w/o bounds and when set to the default 1GB it will eventually die with OOME.  This has happened to me as well as Daniel Ploeg on the mailing list.  In my case, I have 10 RS nodes and OOME happens w/ 1GB heap at only about 30-35 regions per RS.  In previous versions, I have imported to several hundred regions per RS with default heap size.
> I am able to get past this by increasing the max heap to 2GB.  However, the appearance of this in newer versions leads me to believe there is now some kind of memory leak happening in the region servers during import.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.