hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Matteo Bertozzi <theo.berto...@gmail.com>
Subject Re: identifying source of region split request
Date Thu, 07 Jan 2016 03:55:53 GMT
at some point we can use proc-v2 for split/merge HBASE-14551/HBASE-14552
but I'm a bit behind the schedule with the framework.
me and Stephen are working to get in that direction.
but it is probably a month or two away

Matteo


On Wed, Jan 6, 2016 at 7:49 PM, Heng Chen <heng.chen.1986@gmail.com> wrote:

> Could we use Procedure V2 framework to do split?
> It seems to be a great work...
>
> 2016-01-07 4:35 GMT+08:00 Ted Yu <yuzhihong@gmail.com>:
>
> > bq. is splitting and subsequent merging a necessarily bad thing?
> >
> > Please note that merging may happen before the size of R1 (R2) drops to
> > zero. Meaning, there may be churn in normalization activity.
> > I would say that the normalization in the scenario below is premature.
> >
> > Cheers
> >
> > On Wed, Jan 6, 2016 at 11:41 AM, Mikhail Antonov <olorinbant@gmail.com>
> > wrote:
> >
> > > Yeah, I see. Btw, in scenario you described, is splitting and
> subsequent
> > > merging a necessarily bad thing? If data expiration happens not that
> > often,
> > > having more evenly sized regions in between major collections isn't
> worth
> > > it?
> > >
> > > On Wed, Jan 6, 2016 at 10:23 AM, Ted Yu <yuzhihong@gmail.com> wrote:
> > >
> > > > bq. collect statistics to blacklist some RSs with high failure rate
> > > >
> > > > The metadata would help pinpoint the regions which consistently fail
> > > split
> > > > (merge) in the recent past. The failure could be due to corrupt
> > HFile(s)
> > > or
> > > > other reason.
> > > > Having the statistics would also help normalizer avoid the following
> > > > scenario:
> > > >
> > > > region R gets split into R1 and R2
> > > > size of R1 and R2 decreases due to expiration of data
> > > > R1 and R2 get merged into R'
> > > > more data comes into R', resulting in split, again
> > > >
> > > > Cheers
> > > >
> > > > On Wed, Jan 6, 2016 at 10:14 AM, Mikhail Antonov <
> olorinbant@gmail.com
> > >
> > > > wrote:
> > > >
> > > > > Adding this tracing information to know who initiated split is in
> > > general
> > > > > useful thing. Right now though I'm not sure I see how that would
> help
> > > to
> > > > > make better normalization decisions? Region split failure implies
> > > > > underlying FS issue? Any examples/ideas?
> > > > >
> > > > > Kind of..collect statistics to blacklist some RSs with high failure
> > > rate
> > > > > and don't attempt to split regions hosted there in future?
> > > > >
> > > > > On Tue, Jan 5, 2016 at 2:55 PM, Ted Yu <yuzhihong@gmail.com>
> wrote:
> > > > >
> > > > > > Hi,
> > > > > > I recently worked on improving region normalization feature.
> > > > > >
> > > > > > If region split request triggered by the execution of
> > > > > > SplitNormalizationPlan fails, there is no way of knowing whether
> > the
> > > > > failed
> > > > > > split originated from region normalization.
> > > > > > Such association would give RegionNormalizer information so
that
> it
> > > can
> > > > > > make better normalization decisions in the subsequent
> invocations.
> > > > > >
> > > > > > One enhancement I can think of is to embed metadata in
> SplitRequest
> > > > which
> > > > > > gets passed through RegionStateTransitionContext when
> > > > > > RegionServerServices#reportRegionStateTransition() is called.
> > > > > > This way, RegionStateListener can be notified with the metadata
> (id
> > > of
> > > > > the
> > > > > > requester).
> > > > > >
> > > > > > Comment is welcome.
> > > > > >
> > > > >
> > > > >
> > > > >
> > > > > --
> > > > > Thanks,
> > > > > Michael Antonov
> > > > >
> > > >
> > >
> > >
> > >
> > > --
> > > Thanks,
> > > Michael Antonov
> > >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message