Mailing-List: contact user-help@hbase.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@hbase.apache.org
MIME-Version: 1.0
In-Reply-To: 
 <CANZa=GsnYiSK0JKck5ce22jLb2K6Yp+o=gmjCub=WFUuzWFm4A@mail.gmail.com>
References: 
 <CANZa=GtZGNPqeEMvJ5P8RjFV-x93Vnragoymd1FLYc1aHjz1kA@mail.gmail.com>
	<CAHxLZBWk0OpX5=exo4BuzCfh3OzC1q9907XN9_OV1gzx5ESVFw@mail.gmail.com>
	<CAAg3a2qz=kZxkGkmHErHMbG_jc+POy7hyzeDndQDMJ=AYt9n3Q@mail.gmail.com>
	<CAPQV63UhNwdf==0c1gPA5gxycr=eycgTL8UH1kzitmRHG2Y6bA@mail.gmail.com>
	<CANZa=GsnYiSK0JKck5ce22jLb2K6Yp+o=gmjCub=WFUuzWFm4A@mail.gmail.com>
Date: Wed, 20 Apr 2016 17:18:45 -0700
Message-ID: 
 <CAAg3a2qBx34eQerGJOg+vYn-oMPZ2umNi-2m1ei-dfwbryXe3w@mail.gmail.com>
Subject: Re: Retiring empty regions
From: Vladimir Rodionov <vladrodionov@gmail.com>
To: "user@hbase.apache.org" <user@hbase.apache.org>
Content-Type: multipart/alternative; boundary=089e0115f8c64a12770530f3a7f8

--089e0115f8c64a12770530f3a7f8
Content-Type: text/plain; charset=UTF-8

>I'd love to hear your thoughts on this design, Vlad. Maybe you'd like to
>write up a post for the blog? Meanwhile, I'm sure of a couple of us on here
>on the list would appreciate your Cliff's Notes version. I can take this
>into account for my v2 schema design.

Nick, there will be a presentation on time-series HBase (hbasecon.com) Come
join us :)


On Mon, Apr 4, 2016 at 8:34 AM, Nick Dimiduk <ndimiduk@gmail.com> wrote:

> > Crazy idea, but you might be able to take stripped down version of region
> > normalizer code and make a Tool to run? Requesting split or merge is done
> > through the client API, and the only weighing information you need is
> > whether region empty or not, that you could find out too?
>
> Yeah, that's the direction I'm headed.
>
> > A bit off topic, but I think unfortunately region normalizer now ignores
> > empty regions to avoid undoing pre-split on the table.
>
> Unfortunate indeed. Maybe we should be keeping around the initial splits
> list as a metadata attribute on the table?
>
> > With a right row-key design you will never have empty regions due to TTL.
>
> I'd love to hear your thoughts on this design, Vlad. Maybe you'd like to
> write up a post for the blog? Meanwhile, I'm sure of a couple of us on here
> on the list would appreciate your Cliff's Notes version. I can take this
> into account for my v2 schema design.
>
> > So Nick, merge on 1.1 is not recommended??? Was working very well on
> > previous versions. Is ProcV2 really impact it that bad??
>
> How to answer here carefully... I have no reason to believe merge is not
> working on 1.1. I've been on the wrong end of enough "regions stuck in
> transition" support tickets that I'm not keen to put undue stress on my
> master. ProcV2 insures against many scenarios that cause master trauma,
> hence my interest in the implementation details and my preference for
> cluster administration tasks that use it as their source of authority.
>
> Thanks for the thoughts folks.
> -n
>
> On Fri, Apr 1, 2016 at 10:52 AM, Jean-Marc Spaggiari <
> jean-marc@spaggiari.org> wrote:
>
> > ;) That was not the question ;)
> >
> > So Nick, merge on 1.1 is not recommended??? Was working very well on
> > previous versions. Is ProcV2 really impact it that bad??
> >
> > JMS
> >
> > 2016-04-01 13:49 GMT-04:00 Vladimir Rodionov <vladrodionov@gmail.com>:
> >
> > > >> This is something
> > > >> which makes it far less useful for time-series databases with short
> > TTL
> > > on
> > > >> the tables.
> > >
> > > With a right row-key design you will never have empty regions due to
> TTL.
> > >
> > > -Vlad
> > >
> > > On Thu, Mar 31, 2016 at 10:31 PM, Mikhail Antonov <
> olorinbant@gmail.com>
> > > wrote:
> > >
> > > > Crazy idea, but you might be able to take stripped down version of
> > region
> > > > normalizer code and make a Tool to run? Requesting split or merge is
> > done
> > > > through the client API, and the only weighing information you need is
> > > > whether region empty or not, that you could find out too?
> > > >
> > > >
> > > > "Short of upgrading to 1.2 for the region normalizer,"
> > > >
> > > > A bit off topic, but I think unfortunately region normalizer now
> > ignores
> > > > empty regions to avoid undoing pre-split on the table. This is
> > something
> > > > which makes it far less useful for time-series databases with short
> TTL
> > > on
> > > > the tables. We'll need to address that.
> > > >
> > > > -Mikhail
> > > >
> > > > On Thu, Mar 31, 2016 at 9:56 PM, Nick Dimiduk <ndimiduk@gmail.com>
> > > wrote:
> > > >
> > > > > Hi folks,
> > > > >
> > > > > I have a table with TTL enabled. It's been receiving data for a
> while
> > > > > beyond the TTL and I now have a number of empty regions. I'd like
> to
> > > drop
> > > > > those empty regions to free up heap space on the region servers and
> > > > reduce
> > > > > master load. I'm running a 1.1 derivative.
> > > > >
> > > > > The only threads I found on this topic are from circa 0.92
> timeframe.
> > > > >
> > > > > Short of upgrading to 1.2 for the region normalizer, what's the
> > > > recommended
> > > > > method of cleaning up this cruft? Should I be merging empty regions
> > > into
> > > > > their neighbor's? Looks like region merge hasn't been migrated to
> > > ProcV2
> > > > > yet so would be wise to reduce online table activity, or at least
> aim
> > > > for a
> > > > > "quiet period"? Is there a documented process for off-lining and
> > > > deleting a
> > > > > region by name? I don't see anything in the book about it.
> > > > >
> > > > > I experimented with online merge on pseudodist, looks like it's
> > working
> > > > > fine for the most basic case. I'll probably pursue this unless
> > someone
> > > > has
> > > > > some other ideas.
> > > > >
> > > > > Thanks,
> > > > > Nick
> > > > >
> > > >
> > > >
> > > >
> > > > --
> > > > Thanks,
> > > > Michael Antonov
> > > >
> > >
> >
>

--089e0115f8c64a12770530f3a7f8--