hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Varun Sharma <va...@pinterest.com>
Subject Re: Hbase heap size
Date Thu, 17 Jan 2013 23:24:14 GMT
Thanks for the info. I am looking for a balance where I have a write heavy
work load and need excellent read latency. So 40 % to block cache for
caching, 35 % to memstore.

But I would like to also reduce the number of HFiles and amount of
compaction activity. So, having few number of regions and much larger
memstore flush size - like 640M. Could a large memstore flush be a problem
in some sense ? Are updates blocked on memstore flush ? In my case, I would
expect a 600M sized memstore to materialize into a 200-300M sized HFile.

On Thu, Jan 17, 2013 at 2:31 PM, lars hofhansl <larsh@apache.org> wrote:

> A good rule of thumb that I found is to give each region server a Java
> help that is roughly 1/100th of the size of the disk space per region
> server.
> (that is assuming all the default setting: 10G regions, 128M memstores,
> 40% of heap for memstores, 20% of heap for block cache, 3-way replication)
> That is, if you give the region server a 10G heap, you can expect to be
> able to serve about 1T worth of disk space.
> That can be tweaked of course (increase the region size to 20G, if your
> load is mostly readonly you shrink the memstores, etc).
> That way you can reduce that ratio to 1/200 or even less.
> I'm sure other folks will have more detailed input.
> -- Lars
> ________________________________
>  From: Varun Sharma <varun@pinterest.com>
> To: user@hbase.apache.org
> Sent: Thursday, January 17, 2013 1:15 PM
> Subject: Hbase heap size
> Hi,
> I was wondering how much folks typical give to hbase and how much they
> leave for the file system cache for the region server. I am using hbase
> 0.94 and running only the region server and data node daemons. I have a
> system with 15G ram.
> Thanks

  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message