hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Mikhail Antonov <olorinb...@gmail.com>
Subject Re: identifying source of region split request
Date Wed, 06 Jan 2016 19:41:08 GMT
Yeah, I see. Btw, in scenario you described, is splitting and subsequent
merging a necessarily bad thing? If data expiration happens not that often,
having more evenly sized regions in between major collections isn't worth
it?

On Wed, Jan 6, 2016 at 10:23 AM, Ted Yu <yuzhihong@gmail.com> wrote:

> bq. collect statistics to blacklist some RSs with high failure rate
>
> The metadata would help pinpoint the regions which consistently fail split
> (merge) in the recent past. The failure could be due to corrupt HFile(s) or
> other reason.
> Having the statistics would also help normalizer avoid the following
> scenario:
>
> region R gets split into R1 and R2
> size of R1 and R2 decreases due to expiration of data
> R1 and R2 get merged into R'
> more data comes into R', resulting in split, again
>
> Cheers
>
> On Wed, Jan 6, 2016 at 10:14 AM, Mikhail Antonov <olorinbant@gmail.com>
> wrote:
>
> > Adding this tracing information to know who initiated split is in general
> > useful thing. Right now though I'm not sure I see how that would help to
> > make better normalization decisions? Region split failure implies
> > underlying FS issue? Any examples/ideas?
> >
> > Kind of..collect statistics to blacklist some RSs with high failure rate
> > and don't attempt to split regions hosted there in future?
> >
> > On Tue, Jan 5, 2016 at 2:55 PM, Ted Yu <yuzhihong@gmail.com> wrote:
> >
> > > Hi,
> > > I recently worked on improving region normalization feature.
> > >
> > > If region split request triggered by the execution of
> > > SplitNormalizationPlan fails, there is no way of knowing whether the
> > failed
> > > split originated from region normalization.
> > > Such association would give RegionNormalizer information so that it can
> > > make better normalization decisions in the subsequent invocations.
> > >
> > > One enhancement I can think of is to embed metadata in SplitRequest
> which
> > > gets passed through RegionStateTransitionContext when
> > > RegionServerServices#reportRegionStateTransition() is called.
> > > This way, RegionStateListener can be notified with the metadata (id of
> > the
> > > requester).
> > >
> > > Comment is welcome.
> > >
> >
> >
> >
> > --
> > Thanks,
> > Michael Antonov
> >
>



-- 
Thanks,
Michael Antonov

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message