hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Varun Sharma <va...@pinterest.com>
Subject Re: Balancer switch runs causing problems
Date Tue, 28 Jan 2014 04:52:49 GMT
We are not seeing any balancer related logs btw anymore...


On Mon, Jan 27, 2014 at 8:23 PM, Ted Yu <yuzhihong@gmail.com> wrote:

> Looking at the changes since release 0.94.7, I found:
>
> HBASE-8655 Backport to 94 - HBASE-8346(Prefetching .META. rows in case only
> when useCache is set to true)
> HBASE-8698 potential thread creation in MetaScanner.metaScan
>
> If possible, can you upgrade your cluster ?
>
> Cheers
>
>
> On Mon, Jan 27, 2014 at 8:02 PM, Ted Yu <yuzhihong@gmail.com> wrote:
>
> > Do you see the following (from
> > HConnectionManager$HConnectionImplementation#locateRegionInMeta) ?
> >
> >             if (LOG.isDebugEnabled()) {
> >               LOG.debug("locateRegionInMeta parentTable=" +
> >                 Bytes.toString(parentTable) + ", metaLocation=" +
> >                 ((metaLocation == null)? "null": "{" + metaLocation +
> "}")
> > +
> >                 ", attempt=" + tries + " of " +
> >                 this.numRetries + " failed; retrying after sleep of " +
> >
> >
> > On Mon, Jan 27, 2014 at 7:51 PM, Varun Sharma <varun@pinterest.com>
> wrote:
> >
> >> Actually not sometimes but we are always seeing a large # of .META.
> reads
> >> every 5 minutes.
> >>
> >>
> >> On Mon, Jan 27, 2014 at 7:47 PM, Varun Sharma <varun@pinterest.com>
> >> wrote:
> >>
> >> > The default one with 0.94.7... - I dont see any of those logs. Also we
> >> > turned off the balancer switch - but looks like sometimes we still
> see a
> >> > large number of requests to .META. table every 5 minutes.
> >> >
> >> > Varun
> >> >
> >> >
> >> > On Mon, Jan 27, 2014 at 7:37 PM, Ted Yu <yuzhihong@gmail.com> wrote:
> >> >
> >> >> In HMaster#balance(), we have (same for 0.94 and 0.96):
> >> >>
> >> >>         for (RegionPlan plan: plans) {
> >> >>           LOG.info("balance " + plan);
> >> >>
> >> >> Do you see such log in master log ?
> >> >>
> >> >>
> >> >> On Mon, Jan 27, 2014 at 7:26 PM, Varun Sharma <varun@pinterest.com>
> >> >> wrote:
> >> >>
> >> >> > We are seeing one other issue with high read latency (p99 etc.)
on
> >> one
> >> >> of
> >> >> > our read heavy hbase clusters which is correlated with the balancer
> >> >> runs -
> >> >> > every 5 minutes.
> >> >> >
> >> >> > If there is no balancing to do, does the balancer only scan the
> table
> >> >> every
> >> >> > 5 minutes - does it do anything on top of that if the regions
are
> >> >> balanced
> >> >> > ?
> >> >> >
> >> >> > Varun
> >> >> >
> >> >>
> >> >
> >> >
> >>
> >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message