hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ted Yu <yuzhih...@gmail.com>
Subject Re: Using HBase with a shared filesystem (gluster, nfs, s3, etc)
Date Thu, 27 Aug 2015 13:50:04 GMT
How balanced are the table regions in your cluster ?

Cheers

On Thu, Aug 27, 2015 at 6:15 AM, donmai <doodnet@gmail.com> wrote:

> I figured out the issue - the reason wasn't actually region movement taking
> a while, the balancer is actually the thing taking forever:
>
> 2015-08-27 12:50:13,582 DEBUG [hostname,60000,1440642755872-BalancerChore]
> balancer.StochasticLoadBalancer: Could not find a better load balance
> plan.  Tried 0 different configurations in 2211294ms, and did not find
> anything with a computed cost less than 54.18640355329625
>
> After waiting for half an hour to an hour, only one region is ever moved by
> the balancer and this process is repeated. I'm using default settings with
> regard to slop / overall balancing...any idea why it's taking so long?
> Thanks!
>
> On Tue, Aug 25, 2015 at 4:46 PM, anil gupta <anilgupta84@gmail.com> wrote:
>
> > AFAIK, region movement does not moves the data of region on the
> > (distributed)FileSystem. It should only, update metadata of HBase.
> > Did you check diskio stats during region movement?
> >
> > On Tue, Aug 25, 2015 at 10:40 AM, Ted Yu <yuzhihong@gmail.com> wrote:
> >
> > > Please see http://hbase.apache.org/book.html#regions.arch.assignment
> > >
> > > On Tue, Aug 25, 2015 at 10:37 AM, donmai <doodnet@gmail.com> wrote:
> > >
> > > > NFS
> > > > 0.98.10
> > > > Will get to you as soon as I am able, on travel
> > > >
> > > > Is my general understanding correct, though, that there shouldn't be
> > any
> > > > data movement from a region reassignment?
> > > >
> > > > On Tue, Aug 25, 2015 at 12:40 PM, Ted Yu <yuzhihong@gmail.com>
> wrote:
> > > >
> > > > > Can you give a bit more information:
> > > > >
> > > > > which filesystem you use
> > > > > which hbase release you use
> > > > > master log snippet for the long region assignment
> > > > >
> > > > > Thanks
> > > > >
> > > > > On Tue, Aug 25, 2015 at 9:30 AM, donmai <doodnet@gmail.com>
wrote:
> > > > >
> > > > > > Hi,
> > > > > >
> > > > > > I'm curious about how exactly region movement works with regard
> to
> > > data
> > > > > > transfer. To my understanding from the docs given an HDFS-backed
> > > > > cluster, a
> > > > > > region movement / transition involves changing things in meta
> only,
> > > all
> > > > > > data movement for locality is handled by HDFS. In the case where
> > > > rootdir
> > > > > is
> > > > > > a shared file system, there shouldn't be any data movement with
a
> > > > region
> > > > > > reassignment, correct? I'm running into performance issues where
> > > region
> > > > > > assignment takes a very long time and I'm trying to figure out
> why.
> > > > > >
> > > > > > Thanks!
> > > > > >
> > > > >
> > > >
> > >
> >
> >
> >
> > --
> > Thanks & Regards,
> > Anil Gupta
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message