hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Gennady Gilin <gena...@exelate.com>
Subject RE: Hbase cluster configuration
Date Wed, 04 Feb 2009 05:50:20 GMT

-----Original Message-----
From: "Billy Pearson" <sales@pearsonwholesale.com>
To: hbase-user@hadoop.apache.org
Sent: 2/4/2009 4:13 AM
Subject: Re: Hbase cluster configuration

also recommend upgrading to 0.19.0 hadoop/hbase if you can upgrade.


"Andrew Purtell" <apurtell@apache.org> wrote in 
message news:519047.50909.qm@web65513.mail.ac4.yahoo.com...
> Hi Michael,
> I have found that trial and error is necessary now. There are no
> clear formulas. How large the system can scale depends entirely
> on the distributions of various aspects of your data set and on
> the application specific load.
> Cluster start up is the most demanding time. If you have an
> inadequate number of xceivers available in the data node, you
> will see regions fail to deploy at start up with transient
> errors recorded in the master log regarding missing blocks.
> This is an indication you need to increase data node resources.
> I keep a tail of the master log up in a window when the cluster
> is starting up. Add a grep on "ERROR" if you just want to catch
> exceptional conditions. When you see this, increase the number
> of configured xceivers by a factor of two and restart.
> You should also increase the number of handlers in the data
> node configuration to the number of nodes in your cluster.
> Hope this helps,
>   - Andy
>> From: Michael Dagaev
>> Do you know the upper bound for regions per node for 0.18?
>> As I understand, 1000 regions is still OK but 4000 is not.
>> How can I estimate the amount of memory and number of
>> xceivers for 0.18 if I know the key and value size ?

View raw message