hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Samir Ahmic <ahmic.sa...@gmail.com>
Subject Re: hbase-0.94.6.1 balancer issue
Date Fri, 12 Apr 2013 18:06:49 GMT
Hi, J-D

Well at this moment i have that edge case with only one region per table:).
Like i said i was using 0.90 for  long time and regions were distributed
evenly  on all RSs regardless on region per table ratio. Here is
what confused me (like i said i have 2 nodes cluster distributed mode):

start-hbase ------>  tables(regions) are distributed evenlyon two RSs (As
expected)
stop one RS -------> all tables(regions) are moved to remaining RS (as
expected)
start RS that was down ---> run balancer ---> LOG:2013-04-12 19:47:20,725
INFO org.apache.hadoop.hbase.master.LoadBalancer: Skipping load balancing
because balanced cluster; servers=2 regions=1 average=0.5 mostloaded=1
leastloaded=0 ----> all tables(regions) stayed  on one server (this is what
i did not expect ?????) :)

Here is is part of status 'detailed' from shell ater i start RS that was
down and run balancer:

hbase(main):001:0> status 'detailed'
version 0.94.6.1
0 regionsInTransition
master coprocessors: []
2 live servers
    172.17.33.2:60020 1365787755294
        requestsPerSecond=0, numberOfOnlineRegions=0, usedHeapMB=38,
maxHeapMB=3487
    172.17.33.3:60020 1365777858778
        requestsPerSecond=0, numberOfOnlineRegions=49, usedHeapMB=53,
maxHeapMB=3487

So because i have 1 regions per table regions were not rebalances after
start RS  that was down?

Thanks
Samir


On Fri, Apr 12, 2013 at 7:17 PM, Jean-Daniel Cryans <jdcryans@apache.org>wrote:

> Samir,
>
> When you say "And at what point balancer will start redistribute regions to
> second server", do you mean that when you look at the master's web UI you
> see that one region server has 0 region? That would be a problem. Else,
> that line you posted in your original message should be repeated for each
> table, and globally the regions should all be correctly distributed...
> unless there's an edge case where when you have only tables with 1 region
> it puts them all on the same server :)
>
> Thx,
>
> J-D
>
>
> On Fri, Apr 12, 2013 at 12:37 PM, Samir Ahmic <ahmic.samir@gmail.com>
> wrote:
>
> > Thanks for explaining Jean-Marc,
> >
> > We are using 0.90.4 for very long time and balancing was based on total
> > number of regions.That is why i was surprised with balancer log on 0.94.
> > Well i'm more ops guy then dev i handle what other develop :)
> >
> > Regards
> >
> >
> > On Fri, Apr 12, 2013 at 6:24 PM, Jean-Marc Spaggiari <
> > jean-marc@spaggiari.org> wrote:
> >
> > > Hi Samir,
> > >
> > > Since regions are balanced per table, as soon as you will have more
> than
> > > one region in your table, balancer will start to balance the regions
> over
> > > the servers.
> > >
> > > You can split some of those tables and will you start to see HBase
> > balance
> > > them. This is normal behavior for 0.94. I don't know for versions
> before
> > > that.
> > >
> > > Also, are you sure you need 48 tables? And not less tables with more
> CFs?
> > >
> > > JM
> > >
> > > 2013/4/12 Samir Ahmic <ahmic.samir@gmail.com>
> > >
> > > > Hi, JM
> > > >
> > > > I have 48 tables and as you said it is 1 region per table since i did
> > not
> > > > reach splitting limit yet. So this is normal behavior  in 0.94.6.1
> > > version
> > > > ?  And at what point balancer will start redistribute regions to
> second
> > > > server ?
> > > >
> > > > Thanks
> > > > Samir
> > > >
> > > >
> > > > On Fri, Apr 12, 2013 at 6:06 PM, Jean-Marc Spaggiari <
> > > > jean-marc@spaggiari.org> wrote:
> > > >
> > > > > Hi Samir,
> > > > >
> > > > > Regions are balancer per table.
> > > > >
> > > > > So if you have 48 regions within the same table, it should be split
> > > about
> > > > > 24 on each server.
> > > > >
> > > > > But if you have 48 tables with 1 region each, the for each table,
> the
> > > > > balancer will see only 1 region and will display the message you
> saw.
> > > > >
> > > > > Have you looked at the UI? What do you have in it? Can you please
> > > confirm
> > > > > if yo uhave 48 tables or 1 table?
> > > > >
> > > > > Thanks,
> > > > >
> > > > > JM
> > > > >
> > > > >
> > > > > 2013/4/12 Samir Ahmic <ahmic.samir@gmail.com>
> > > > >
> > > > > > Hi, all
> > > > > >
> > > > > > I'm evaluating hbase-0.94.6.1 and i have 48 regions on 2 node
> > > cluster.
> > > > I
> > > > > > was restarting on of RSs and after that tried to balance cluster
> by
> > > > > running
> > > > > > balancer from shell. After running command regions were not
> > > distributed
> > > > > to
> > > > > > second RS and i found this line i master log:
> > > > > >
> > > > > > 2013-04-12 16:45:15,589 INFO
> > > > org.apache.hadoop.hbase.master.LoadBalancer:
> > > > > > Skipping load balancing because balanced cluster; servers=2
> > > *regions=1
> > > > > > *average=0.5
> > > > > > mostloaded=1 leastloaded=0
> > > > > >
> > > > > > This look like to me that wrong number of regions is reported
by
> > > > balancer
> > > > > > and that cause of  skipping load balancing . In hbase shell
i see
> > all
> > > > 48
> > > > > > tables that i have and everything else looks fine.
> > > > > >
> > > > > > Did someone else see this type of behavior ? Did something
> changed
> > > > around
> > > > > > balancer in hbase-0.94.6.1 ?
> > > > > >
> > > > > > Regards
> > > > > > Samir
> > > > > >
> > > > >
> > > >
> > >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message