hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Nick Dimiduk <ndimi...@gmail.com>
Subject Re: Retiring empty regions
Date Mon, 04 Apr 2016 15:34:06 GMT
> Crazy idea, but you might be able to take stripped down version of region
> normalizer code and make a Tool to run? Requesting split or merge is done
> through the client API, and the only weighing information you need is
> whether region empty or not, that you could find out too?

Yeah, that's the direction I'm headed.

> A bit off topic, but I think unfortunately region normalizer now ignores
> empty regions to avoid undoing pre-split on the table.

Unfortunate indeed. Maybe we should be keeping around the initial splits
list as a metadata attribute on the table?

> With a right row-key design you will never have empty regions due to TTL.

I'd love to hear your thoughts on this design, Vlad. Maybe you'd like to
write up a post for the blog? Meanwhile, I'm sure of a couple of us on here
on the list would appreciate your Cliff's Notes version. I can take this
into account for my v2 schema design.

> So Nick, merge on 1.1 is not recommended??? Was working very well on
> previous versions. Is ProcV2 really impact it that bad??

How to answer here carefully... I have no reason to believe merge is not
working on 1.1. I've been on the wrong end of enough "regions stuck in
transition" support tickets that I'm not keen to put undue stress on my
master. ProcV2 insures against many scenarios that cause master trauma,
hence my interest in the implementation details and my preference for
cluster administration tasks that use it as their source of authority.

Thanks for the thoughts folks.
-n

On Fri, Apr 1, 2016 at 10:52 AM, Jean-Marc Spaggiari <
jean-marc@spaggiari.org> wrote:

> ;) That was not the question ;)
>
> So Nick, merge on 1.1 is not recommended??? Was working very well on
> previous versions. Is ProcV2 really impact it that bad??
>
> JMS
>
> 2016-04-01 13:49 GMT-04:00 Vladimir Rodionov <vladrodionov@gmail.com>:
>
> > >> This is something
> > >> which makes it far less useful for time-series databases with short
> TTL
> > on
> > >> the tables.
> >
> > With a right row-key design you will never have empty regions due to TTL.
> >
> > -Vlad
> >
> > On Thu, Mar 31, 2016 at 10:31 PM, Mikhail Antonov <olorinbant@gmail.com>
> > wrote:
> >
> > > Crazy idea, but you might be able to take stripped down version of
> region
> > > normalizer code and make a Tool to run? Requesting split or merge is
> done
> > > through the client API, and the only weighing information you need is
> > > whether region empty or not, that you could find out too?
> > >
> > >
> > > "Short of upgrading to 1.2 for the region normalizer,"
> > >
> > > A bit off topic, but I think unfortunately region normalizer now
> ignores
> > > empty regions to avoid undoing pre-split on the table. This is
> something
> > > which makes it far less useful for time-series databases with short TTL
> > on
> > > the tables. We'll need to address that.
> > >
> > > -Mikhail
> > >
> > > On Thu, Mar 31, 2016 at 9:56 PM, Nick Dimiduk <ndimiduk@gmail.com>
> > wrote:
> > >
> > > > Hi folks,
> > > >
> > > > I have a table with TTL enabled. It's been receiving data for a while
> > > > beyond the TTL and I now have a number of empty regions. I'd like to
> > drop
> > > > those empty regions to free up heap space on the region servers and
> > > reduce
> > > > master load. I'm running a 1.1 derivative.
> > > >
> > > > The only threads I found on this topic are from circa 0.92 timeframe.
> > > >
> > > > Short of upgrading to 1.2 for the region normalizer, what's the
> > > recommended
> > > > method of cleaning up this cruft? Should I be merging empty regions
> > into
> > > > their neighbor's? Looks like region merge hasn't been migrated to
> > ProcV2
> > > > yet so would be wise to reduce online table activity, or at least aim
> > > for a
> > > > "quiet period"? Is there a documented process for off-lining and
> > > deleting a
> > > > region by name? I don't see anything in the book about it.
> > > >
> > > > I experimented with online merge on pseudodist, looks like it's
> working
> > > > fine for the most basic case. I'll probably pursue this unless
> someone
> > > has
> > > > some other ideas.
> > > >
> > > > Thanks,
> > > > Nick
> > > >
> > >
> > >
> > >
> > > --
> > > Thanks,
> > > Michael Antonov
> > >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message