Mailing-List: contact user-help@hbase.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@hbase.apache.org
MIME-Version: 1.0
In-Reply-To: 
 <CAAg3a2qBx34eQerGJOg+vYn-oMPZ2umNi-2m1ei-dfwbryXe3w@mail.gmail.com>
References: 
 <CANZa=GtZGNPqeEMvJ5P8RjFV-x93Vnragoymd1FLYc1aHjz1kA@mail.gmail.com>
 <CAHxLZBWk0OpX5=exo4BuzCfh3OzC1q9907XN9_OV1gzx5ESVFw@mail.gmail.com>
 <CAAg3a2qz=kZxkGkmHErHMbG_jc+POy7hyzeDndQDMJ=AYt9n3Q@mail.gmail.com>
 <CAPQV63UhNwdf==0c1gPA5gxycr=eycgTL8UH1kzitmRHG2Y6bA@mail.gmail.com>
 <CANZa=GsnYiSK0JKck5ce22jLb2K6Yp+o=gmjCub=WFUuzWFm4A@mail.gmail.com>
 <CAAg3a2qBx34eQerGJOg+vYn-oMPZ2umNi-2m1ei-dfwbryXe3w@mail.gmail.com>
From: Nick Dimiduk <ndimiduk@gmail.com>
Date: Tue, 26 Apr 2016 09:16:38 -0700
Message-ID: 
 <CANZa=Gv1XuoL3cgoKZB+e37aUX6PJS9Z85z+Lg_yxAxHr2L4PA@mail.gmail.com>
Subject: Re: Retiring empty regions
To: hbase-user <user@hbase.apache.org>
Content-Type: multipart/alternative; boundary=001a1141e52c5664900531659f96

--001a1141e52c5664900531659f96
Content-Type: text/plain; charset=UTF-8

I'm looking forward to your talk Vlad.

In the mean time, I filed HBASE-15712. We'll get our implementation posted
up there. We have these deployed on one of the masters, running daily with
cron.

@Mikhail, to get this feature into the normalizer, how about this: let's
add a min number of regions property to user tables. This can be set when
someone creates a table with split points, or maintained manually. The
normalizer can use that as a constraint to guide its convergence.

On Wed, Apr 20, 2016 at 5:18 PM, Vladimir Rodionov <vladrodionov@gmail.com>
wrote:

> >I'd love to hear your thoughts on this design, Vlad. Maybe you'd like to
> >write up a post for the blog? Meanwhile, I'm sure of a couple of us on
> here
> >on the list would appreciate your Cliff's Notes version. I can take this
> >into account for my v2 schema design.
>
> Nick, there will be a presentation on time-series HBase (hbasecon.com)
> Come
> join us :)
>
>
> On Mon, Apr 4, 2016 at 8:34 AM, Nick Dimiduk <ndimiduk@gmail.com> wrote:
>
> > > Crazy idea, but you might be able to take stripped down version of
> region
> > > normalizer code and make a Tool to run? Requesting split or merge is
> done
> > > through the client API, and the only weighing information you need is
> > > whether region empty or not, that you could find out too?
> >
> > Yeah, that's the direction I'm headed.
> >
> > > A bit off topic, but I think unfortunately region normalizer now
> ignores
> > > empty regions to avoid undoing pre-split on the table.
> >
> > Unfortunate indeed. Maybe we should be keeping around the initial splits
> > list as a metadata attribute on the table?
> >
> > > With a right row-key design you will never have empty regions due to
> TTL.
> >
> > I'd love to hear your thoughts on this design, Vlad. Maybe you'd like to
> > write up a post for the blog? Meanwhile, I'm sure of a couple of us on
> here
> > on the list would appreciate your Cliff's Notes version. I can take this
> > into account for my v2 schema design.
> >
> > > So Nick, merge on 1.1 is not recommended??? Was working very well on
> > > previous versions. Is ProcV2 really impact it that bad??
> >
> > How to answer here carefully... I have no reason to believe merge is not
> > working on 1.1. I've been on the wrong end of enough "regions stuck in
> > transition" support tickets that I'm not keen to put undue stress on my
> > master. ProcV2 insures against many scenarios that cause master trauma,
> > hence my interest in the implementation details and my preference for
> > cluster administration tasks that use it as their source of authority.
> >
> > Thanks for the thoughts folks.
> > -n
> >
> > On Fri, Apr 1, 2016 at 10:52 AM, Jean-Marc Spaggiari <
> > jean-marc@spaggiari.org> wrote:
> >
> > > ;) That was not the question ;)
> > >
> > > So Nick, merge on 1.1 is not recommended??? Was working very well on
> > > previous versions. Is ProcV2 really impact it that bad??
> > >
> > > JMS
> > >
> > > 2016-04-01 13:49 GMT-04:00 Vladimir Rodionov <vladrodionov@gmail.com>:
> > >
> > > > >> This is something
> > > > >> which makes it far less useful for time-series databases with
> short
> > > TTL
> > > > on
> > > > >> the tables.
> > > >
> > > > With a right row-key design you will never have empty regions due to
> > TTL.
> > > >
> > > > -Vlad
> > > >
> > > > On Thu, Mar 31, 2016 at 10:31 PM, Mikhail Antonov <
> > olorinbant@gmail.com>
> > > > wrote:
> > > >
> > > > > Crazy idea, but you might be able to take stripped down version of
> > > region
> > > > > normalizer code and make a Tool to run? Requesting split or merge
> is
> > > done
> > > > > through the client API, and the only weighing information you need
> is
> > > > > whether region empty or not, that you could find out too?
> > > > >
> > > > >
> > > > > "Short of upgrading to 1.2 for the region normalizer,"
> > > > >
> > > > > A bit off topic, but I think unfortunately region normalizer now
> > > ignores
> > > > > empty regions to avoid undoing pre-split on the table. This is
> > > something
> > > > > which makes it far less useful for time-series databases with short
> > TTL
> > > > on
> > > > > the tables. We'll need to address that.
> > > > >
> > > > > -Mikhail
> > > > >
> > > > > On Thu, Mar 31, 2016 at 9:56 PM, Nick Dimiduk <ndimiduk@gmail.com>
> > > > wrote:
> > > > >
> > > > > > Hi folks,
> > > > > >
> > > > > > I have a table with TTL enabled. It's been receiving data for a
> > while
> > > > > > beyond the TTL and I now have a number of empty regions. I'd like
> > to
> > > > drop
> > > > > > those empty regions to free up heap space on the region servers
> and
> > > > > reduce
> > > > > > master load. I'm running a 1.1 derivative.
> > > > > >
> > > > > > The only threads I found on this topic are from circa 0.92
> > timeframe.
> > > > > >
> > > > > > Short of upgrading to 1.2 for the region normalizer, what's the
> > > > > recommended
> > > > > > method of cleaning up this cruft? Should I be merging empty
> regions
> > > > into
> > > > > > their neighbor's? Looks like region merge hasn't been migrated to
> > > > ProcV2
> > > > > > yet so would be wise to reduce online table activity, or at least
> > aim
> > > > > for a
> > > > > > "quiet period"? Is there a documented process for off-lining and
> > > > > deleting a
> > > > > > region by name? I don't see anything in the book about it.
> > > > > >
> > > > > > I experimented with online merge on pseudodist, looks like it's
> > > working
> > > > > > fine for the most basic case. I'll probably pursue this unless
> > > someone
> > > > > has
> > > > > > some other ideas.
> > > > > >
> > > > > > Thanks,
> > > > > > Nick
> > > > > >
> > > > >
> > > > >
> > > > >
> > > > > --
> > > > > Thanks,
> > > > > Michael Antonov
> > > > >
> > > >
> > >
> >
>

--001a1141e52c5664900531659f96--