hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Bryan Beaudreault <bbeaudrea...@hubspot.com>
Subject Re: Unbalanced region server
Date Tue, 24 Apr 2012 03:43:22 GMT
Over time the balancer will move around both existing and new regions. A few questions: How
many tables do you have? Are they all configured similarly? What version of hbase are you
on?

Sent from iPhone.

On Apr 23, 2012, at 11:39 PM, David Charle <dbchar2012@gmail.com> wrote:

> Does balancer_switch=true; rebalances any skew in the existing keys or it
> only does for all new regions ? The requests seems to be oK across all
> servers.
> 
> On Mon, Apr 23, 2012 at 8:03 PM, David Charle <dbchar2012@gmail.com> wrote:
> 
>> Thanks Bryan.
>> 
>> Here is the regions count:
>> 
>> 1: 2616
>> 2: 2620
>> 3: 2623
>> 4: 2617
>> 5: 2617
>> 
>> The skew is in Node 2 where we have the space issue (double the size of
>> other 4 nodes).
>> 
>> 
>> On Mon, Apr 23, 2012 at 7:50 PM, Bryan Beaudreault <
>> bbeaudreault@hubspot.com> wrote:
>> 
>>> Do all nodes have the same number of regions?  If not you may not have the
>>> balancer on.  You can turn it on using balancer_switch true in the hbase
>>> shell.  The balancer also doesn't run if there is a region stuck in
>>> transition.  In this case if your data is growing rapidly I have seen
>>> regionservers become lopsided until I could clean up the stuck region and
>>> re-enable the balancer.
>>> 
>>> Finally, depending on your version of hbase, the balancer only tries to
>>> keep the number of regions similar across all regionservers.  If you have
>>> tables of different max region sizes I could imagine a case where one
>>> region server unluckily is hosting more regions from the table with larger
>>> region sizes.  This also might explain the imbalance in requests, if one
>>> of
>>> your tables gets more traffic than other.  The HMaster UI would be helpful
>>> for determining the spread of regions per table across regionservers.
>>> 
>>> On Mon, Apr 23, 2012 at 9:41 PM, David Charle <dbchar2012@gmail.com>
>>> wrote:
>>> 
>>>> Hi
>>>> 
>>>> What can make a region server unbalanced when it comes to space ? (and
>>>> possibly requests too).
>>>> 
>>>> For example; I have 5 node cluster (replication factor of 3); and in
>>> which
>>>> all 4 has same size; where as one node is almost double in size for
>>> /hbase.
>>>> 
>>>> Any help will be appreciated; esp what makes the skew and how to fix it
>>> ?
>>>> 
>>>> Thanks
>>>> David
>>>> 
>>> 
>> 
>> 

Mime
View raw message