hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jonathan Gray" <jl...@streamy.com>
Subject RE: thinking about hbase 0.20
Date Thu, 02 Apr 2009 20:23:31 GMT
I personally feel very strongly about the need to finish all things
surrounding 1249.

Erik and I have spent an enormous amount of time designing and
re-implementing the client, api, implementation of gets/puts/deletes, etc...

Without these things HBase will be improved but will still be doing all
sorts of silly things in implementation that cause problems with high
numbers of columns, poor performance on deletes, and basically never taking
advantage of "early-out" scenarios requiring entire scans in almost every
case today.

The good news is that it's mostly done.  We're waiting to get a solid 1234
patch committed and tested before breaking it apart again.  It's significant
change but well thought out and mostly complete.  Now is the time to make
these more radical changes, there's a full migration either way.

JG

> -----Original Message-----
> From: saint.ack@gmail.com [mailto:saint.ack@gmail.com] On Behalf Of
> stack
> Sent: Wednesday, April 01, 2009 11:42 PM
> To: hbase-dev@hadoop.apache.org
> Subject: Re: thinking about hbase 0.20
> 
> That'd be ideal.
> 
> Regards what features should be in 0.20.0, we should start in weeding
> the
> list of 77 issues currently filed against 0.20.0 here:
> https://issues.apache.org/jira/secure/IssueNavigator.jspa?reset=true&mo
> de=hide&sorter/order=DESC&sorter/field=priority&resolution=-
> 1&pid=12310753&fixfor=12313474
> .
> 
> St.Ack
> 
> On Thu, Apr 2, 2009 at 9:13 AM, Ryan Rawson <ryanobjc@gmail.com> wrote:
> 
> > Thinking about a migration, practically speaking this would be
> doable:
> >
> > - Flush and compact everything.  Get rid of reference files from
> region
> > splits.
> > - Take each mapfile (ignore the index files), read the file in, write
> an
> > equivalent hfile out.
> > - Done!
> >
> > This can't be done while the cluster is online however.
> >
> > Any other suggestions?
> >
> > On Thu, Apr 2, 2009 at 12:09 AM, stack <stack@duboce.net> wrote:
> >
> > > I made HBASE-1215 as issue to cover migration from 0.19.x to
> 0.20.0.
> > >
> > > We have a migration 'system' already.  You run ./bin/hbase migrate.
> Going
> > > from 0.19.0 to 0.20.0, we'll need to add a mapreduce job that
> rewrites
> > all
> > > hbase data to new format.  It needs to be MR for those cases where
> data
> > is
> > > large.
> > >
> > > I thought at first that we could do lazy migration but after
> looking at
> > it,
> > > keeping up two key types in the one context looked too complex.
> > >
> > > St.Ack
> > >
> > > On Thu, Apr 2, 2009 at 8:51 AM, Ryan Rawson <ryanobjc@gmail.com>
> wrote:
> > >
> > > > hi all,
> > > >
> > > > it's been a long road, but it's time to start thinking about what
> will
> > > > conclusively be in 0.20.
> > > >
> > > > I'll let you fight that out a bit... personally I'd be happy with
> hfile
> > +
> > > > KeyValue.
> > > >
> > > > But, one last thing, what is our migration story going to be?
> > > >
> > > > -ryan
> > > >
> > >
> >


Mime
View raw message