hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "stack (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HBASE-900) Regionserver memory leak causing OOME during relatively modest bulk importing
Date Sat, 13 Dec 2008 01:16:46 GMT

    [ https://issues.apache.org/jira/browse/HBASE-900?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12656221#action_12656221

stack commented on HBASE-900:

Applying v7.     Can improve on it in later patches as get more info. 

+ Downs the client-side batch write heap default from 10MB to 2MB
+ Adds ByteSize interface with a heapSize member.  BatchUpdate, HStoreKey, etc., implement
+ The sizes returned out of heapSize favor 64-bit JVMs.  Were obtained from study of heaps
made by running HRS and from runs of the new BU and Memcache mains which have little scripts
to generate heaps with arrays of BUs and different Memcaches which can then be heap-dumped
and studied.

> Regionserver memory leak causing OOME during relatively modest bulk importing
> -----------------------------------------------------------------------------
>                 Key: HBASE-900
>                 URL: https://issues.apache.org/jira/browse/HBASE-900
>             Project: Hadoop HBase
>          Issue Type: Bug
>    Affects Versions: 0.18.1, 0.19.0
>            Reporter: Jonathan Gray
>            Assignee: stack
>            Priority: Blocker
>         Attachments: 900-part2-v4.patch, 900-part2-v5.patch, 900-part2-v7.patch, 900-part2.patch,
900.patch, memoryOn13.png
> I have recreated this issue several times and it appears to have been introduced in 0.2.
> During an import to a single table, memory usage of individual region servers grows w/o
bounds and when set to the default 1GB it will eventually die with OOME.  This has happened
to me as well as Daniel Ploeg on the mailing list.  In my case, I have 10 RS nodes and OOME
happens w/ 1GB heap at only about 30-35 regions per RS.  In previous versions, I have imported
to several hundred regions per RS with default heap size.
> I am able to get past this by increasing the max heap to 2GB.  However, the appearance
of this in newer versions leads me to believe there is now some kind of memory leak happening
in the region servers during import.

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message