hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Anil Gupta (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-9399) Up the memstore flush size
Date Thu, 19 Sep 2013 03:26:53 GMT

    [ https://issues.apache.org/jira/browse/HBASE-9399?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13771556#comment-13771556

Anil Gupta commented on HBASE-9399:

[~eclark] I am also thinking of upping the flush size from 128 mb since i have ~30gb heap.
If you have got the update on perf numbers. It would be great if you can share them.
> Up the memstore flush size
> --------------------------
>                 Key: HBASE-9399
>                 URL: https://issues.apache.org/jira/browse/HBASE-9399
>             Project: HBase
>          Issue Type: Task
>          Components: regionserver
>    Affects Versions: 0.98.0, 0.96.0
>            Reporter: Elliott Clark
>            Assignee: Elliott Clark
> As heap sizes get bigger we are still recommending that users keep their number of regions
to a minimum.  This leads to lots of un-used memstore memory.
> For example I have a region server with 48 gigs of ram.  30 gigs are there for the region
server.  This with current defaults the global memstore size reserved is 8 gigs.
> The per region memstore size is 128mb right now.  That means that I need 80 regions actively
taking writes to reach the global memstore size.  That number is way out of line with what
our split policies currently give users.  They are given much fewer regions by default.
> We should up the hbase.hregion.memstore.flush.size size.  Ideally we should auto tune
everything.  But until then I think something like 512mb would help a lot with our write throughput
on clusters that don't have several hundred regions per RS.

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

View raw message