hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jean-Daniel Cryans <jdcry...@apache.org>
Subject Re: Hardware and Heap Allocation Questions
Date Thu, 19 Jul 2012 18:42:13 GMT
Inline.

J-D

On Tue, Jul 17, 2012 at 2:49 PM, Young Kim <ykim@crunchyroll.com> wrote:
> 1. There seems to be a dearth of articles in what hardware most people use for their
master servers. So, is there a recommended spec for the Master servers?

The HBase master in particular doesn't need anything special, it's
your NameNodes you might want to make more reliable until you are
using HA NN in 2.0

> 2. After some research, it seems that the normal for Region servers are lots of RAM (24GB+),
12 spindle setups, and 2 x 6 cores. However, I recall reading some articles warning against
the allocation of more then 16GB to Regions. Is this warning still valid, or do workarounds
exist?

It's a case of YMMV. We've been running with 25GB on our user-facing
cluster for almost a year now and it never caused issues. We almost
never run big inserts that put a lot of memory pressure (or any MR job
that transit a lot of data in HBase, we have another cluster for
that), so we never hit Full GCs.

> 3. Finally, is there an article on the recommended heap allocation for each processes?

Mmm I guess there's a Cloudera article that talks about it, but
generally you don't need to give more than 1GB (default) or 2 to the
TaskTrackers, Datanodes and HMasters. Give everything you can to the
Namenode. If you don't run MR jobs then give a lot of RAM to the
region servers or split the memory between the tasks and the region
servers.

Hope this helps,

J-D

Mime
View raw message