hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ryan Rawson <ryano...@gmail.com>
Subject Re: setting of HBase, how much memory should be reserved for a Hbase regionserver.
Date Mon, 01 Mar 2010 08:08:08 GMT
I would consider doing the following things:

- disable the block cache - you dont have ram to spare. 4gb is really
minimum (my laptop is 4gb ram and i decry how sucky I am occasionally)
- run between 1000-2000m, with the cache disabled you are in better shape
- consider adjusting the config values:
    <name>hbase.regionserver.global.memstore.upperLimit</name>
    <value>0.4</value>

This implies that memstore (the write buffer) can occupy 40% of heap.
This might be too aggressive with a small heap and CMS.  Try changing
that to 0.2 and:

    <name>hbase.regionserver.global.memstore.lowerLimit</name>
    <value>0.35</value>

to 0.10 or so.

Good luck!
-ryan

On Sun, Feb 28, 2010 at 11:59 PM, steven zhuang <zhuangxin8448@gmail.com> wrote:
> hi, all,
>          We have some PCs(a dozens of them) with  4 cores CPU and 4GB ram
> each, a hadoop instance is already running on these machines.
>
>          Currently there is around 1.2GB ram left on every node.
>
>          We want to setup a Hbase instance on these machines, with 3 quorum
> servers and 5 region servers, but we kept getting "out of memory" error when
> the table grows big on some nodes.
>
>          I know HBase is really a memory eater, my question is can we set
> up a Hbase cluster on these PC if we add more region servers? how much ram
> should we reserve to make hbase run happily?
>
>          An option is we put hbase regionservers and Hadoop
> datanode/tasktracker on different node respectively(Not tried yet), but I
> don't know how much this would influence the performance, as we can not
> benefit from the data locality.
>
> --
>       best wishes.
>                            steven
>

Mime
View raw message