hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ted Yu <yuzhih...@gmail.com>
Subject Re: identifying source of region split request
Date Wed, 06 Jan 2016 20:35:42 GMT
bq. is splitting and subsequent merging a necessarily bad thing?

Please note that merging may happen before the size of R1 (R2) drops to
zero. Meaning, there may be churn in normalization activity.
I would say that the normalization in the scenario below is premature.

Cheers

On Wed, Jan 6, 2016 at 11:41 AM, Mikhail Antonov <olorinbant@gmail.com>
wrote:

> Yeah, I see. Btw, in scenario you described, is splitting and subsequent
> merging a necessarily bad thing? If data expiration happens not that often,
> having more evenly sized regions in between major collections isn't worth
> it?
>
> On Wed, Jan 6, 2016 at 10:23 AM, Ted Yu <yuzhihong@gmail.com> wrote:
>
> > bq. collect statistics to blacklist some RSs with high failure rate
> >
> > The metadata would help pinpoint the regions which consistently fail
> split
> > (merge) in the recent past. The failure could be due to corrupt HFile(s)
> or
> > other reason.
> > Having the statistics would also help normalizer avoid the following
> > scenario:
> >
> > region R gets split into R1 and R2
> > size of R1 and R2 decreases due to expiration of data
> > R1 and R2 get merged into R'
> > more data comes into R', resulting in split, again
> >
> > Cheers
> >
> > On Wed, Jan 6, 2016 at 10:14 AM, Mikhail Antonov <olorinbant@gmail.com>
> > wrote:
> >
> > > Adding this tracing information to know who initiated split is in
> general
> > > useful thing. Right now though I'm not sure I see how that would help
> to
> > > make better normalization decisions? Region split failure implies
> > > underlying FS issue? Any examples/ideas?
> > >
> > > Kind of..collect statistics to blacklist some RSs with high failure
> rate
> > > and don't attempt to split regions hosted there in future?
> > >
> > > On Tue, Jan 5, 2016 at 2:55 PM, Ted Yu <yuzhihong@gmail.com> wrote:
> > >
> > > > Hi,
> > > > I recently worked on improving region normalization feature.
> > > >
> > > > If region split request triggered by the execution of
> > > > SplitNormalizationPlan fails, there is no way of knowing whether the
> > > failed
> > > > split originated from region normalization.
> > > > Such association would give RegionNormalizer information so that it
> can
> > > > make better normalization decisions in the subsequent invocations.
> > > >
> > > > One enhancement I can think of is to embed metadata in SplitRequest
> > which
> > > > gets passed through RegionStateTransitionContext when
> > > > RegionServerServices#reportRegionStateTransition() is called.
> > > > This way, RegionStateListener can be notified with the metadata (id
> of
> > > the
> > > > requester).
> > > >
> > > > Comment is welcome.
> > > >
> > >
> > >
> > >
> > > --
> > > Thanks,
> > > Michael Antonov
> > >
> >
>
>
>
> --
> Thanks,
> Michael Antonov
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message