Mailing-List: contact dev-help@hbase.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@hbase.apache.org
MIME-Version: 1.0
In-Reply-To: 
 <CAHxLZBVAbHqd8Rb14rfrhChHYno6oq-Yu3r2Fa-fYuVCKN4AeQ@mail.gmail.com>
References: 
 <CALte62yaqW3g_S_U8bDMO9=N0H-7+-Yib7s+FswidRE_qE3uqw@mail.gmail.com>
	<CAHxLZBVAbHqd8Rb14rfrhChHYno6oq-Yu3r2Fa-fYuVCKN4AeQ@mail.gmail.com>
Date: Wed, 6 Jan 2016 10:23:11 -0800
Message-ID: 
 <CALte62wksYU6TfufybvNDZ8V1AJtAoYWWiHF+H1N=W-0fbx-OA@mail.gmail.com>
Subject: Re: identifying source of region split request
From: Ted Yu <yuzhihong@gmail.com>
To: "dev@hbase.apache.org" <dev@hbase.apache.org>
Content-Type: multipart/alternative; boundary=001a114299aa53dfa70528ae72eb

--001a114299aa53dfa70528ae72eb
Content-Type: text/plain; charset=UTF-8

bq. collect statistics to blacklist some RSs with high failure rate

The metadata would help pinpoint the regions which consistently fail split
(merge) in the recent past. The failure could be due to corrupt HFile(s) or
other reason.
Having the statistics would also help normalizer avoid the following
scenario:

region R gets split into R1 and R2
size of R1 and R2 decreases due to expiration of data
R1 and R2 get merged into R'
more data comes into R', resulting in split, again

Cheers

On Wed, Jan 6, 2016 at 10:14 AM, Mikhail Antonov <olorinbant@gmail.com>
wrote:

> Adding this tracing information to know who initiated split is in general
> useful thing. Right now though I'm not sure I see how that would help to
> make better normalization decisions? Region split failure implies
> underlying FS issue? Any examples/ideas?
>
> Kind of..collect statistics to blacklist some RSs with high failure rate
> and don't attempt to split regions hosted there in future?
>
> On Tue, Jan 5, 2016 at 2:55 PM, Ted Yu <yuzhihong@gmail.com> wrote:
>
> > Hi,
> > I recently worked on improving region normalization feature.
> >
> > If region split request triggered by the execution of
> > SplitNormalizationPlan fails, there is no way of knowing whether the
> failed
> > split originated from region normalization.
> > Such association would give RegionNormalizer information so that it can
> > make better normalization decisions in the subsequent invocations.
> >
> > One enhancement I can think of is to embed metadata in SplitRequest which
> > gets passed through RegionStateTransitionContext when
> > RegionServerServices#reportRegionStateTransition() is called.
> > This way, RegionStateListener can be notified with the metadata (id of
> the
> > requester).
> >
> > Comment is welcome.
> >
>
>
>
> --
> Thanks,
> Michael Antonov
>

--001a114299aa53dfa70528ae72eb--