hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Billy Pearson" <sa...@pearsonwholesale.com>
Subject Re: Hbase cluster configuration
Date Wed, 04 Feb 2009 02:13:04 GMT
also recommend upgrading to 0.19.0 hadoop/hbase if you can upgrade.


"Andrew Purtell" <apurtell@apache.org> wrote in 
message news:519047.50909.qm@web65513.mail.ac4.yahoo.com...
> Hi Michael,
> I have found that trial and error is necessary now. There are no
> clear formulas. How large the system can scale depends entirely
> on the distributions of various aspects of your data set and on
> the application specific load.
> Cluster start up is the most demanding time. If you have an
> inadequate number of xceivers available in the data node, you
> will see regions fail to deploy at start up with transient
> errors recorded in the master log regarding missing blocks.
> This is an indication you need to increase data node resources.
> I keep a tail of the master log up in a window when the cluster
> is starting up. Add a grep on "ERROR" if you just want to catch
> exceptional conditions. When you see this, increase the number
> of configured xceivers by a factor of two and restart.
> You should also increase the number of handlers in the data
> node configuration to the number of nodes in your cluster.
> Hope this helps,
>   - Andy
>> From: Michael Dagaev
>> Do you know the upper bound for regions per node for 0.18?
>> As I understand, 1000 regions is still OK but 4000 is not.
>> How can I estimate the amount of memory and number of
>> xceivers for 0.18 if I know the key and value size ?

View raw message