hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ted Yu <yuzhih...@gmail.com>
Subject Re: hbase 0.94.0
Date Mon, 30 Jan 2012 19:46:17 GMT
Before long term wire and format compatibility is reached, I am -0 on time
based release.

As Nicolas pointed out, FB wants to migrate from 0.89-fb to 0.94
If we do time-based releases without wire and format compatibility, it
would consume a lot of our energy satisfying such requirements from
different companies.

I propose wire and format compatibility as the main theme for 0.96

Cheers

On Mon, Jan 30, 2012 at 11:40 AM, Andrew Purtell <apurtell@apache.org>wrote:

> > Maybe the discussion should be whether 0.94 should be a time-gated or
> > feature-gated release? IMO we already have enough good stuff in there
> > on the perf front. It will be great if the checksum improvement makes
> > it, but if it doesn't, I'd rather have a release than delay for it.
>
>
> Time gated releases makes sense going forward for a few reasons, mark me
> down for that option.
>
>
> Best regards,
>
>     - Andy
>
> Problems worthy of attack prove their worth by hitting back. - Piet Hein
> (via Tom White)
>
>
> ----- Original Message -----
> > From: Todd Lipcon <todd@cloudera.com>
> > To: dev@hbase.apache.org
> > Cc: lars hofhansl <lhofhansl@yahoo.com>; Matt Corgan <
> mcorgan@hotpads.com>
> > Sent: Monday, January 30, 2012 10:44 AM
> > Subject: Re: hbase 0.94.0
> >
> > On Sun, Jan 29, 2012 at 7:30 AM, Ted Yu <yuzhihong@gmail.com> wrote:
> >>  Initial results from HBASE-5074, support checksums in HBase block
> cache,
> >>  look promising.
> >>
> >>  Shall we discuss whether 0.94 should include it (assuming Dhruba can
> finish
> >>  the feature in Feb.) ?
> >
> > If it's a time-based release, then there's nothing to discuss. Either
> > it's done in time, in which case it's part of 94, or it's not, in
> > which case it's part of 96, right?
> >
> > Maybe the discussion should be whether 0.94 should be a time-gated or
> > feature-gated release? IMO we already have enough good stuff in there
> > on the perf front. It will be great if the checksum improvement makes
> > it, but if it doesn't, I'd rather have a release than delay for it.
> >
> > -Todd
> >
> >>
> >>  Cheers
> >>
> >>  On Fri, Jan 27, 2012 at 6:27 PM, lars hofhansl <lhofhansl@yahoo.com>
> > wrote:
> >>
> >>>  Salesforce will jumpstart with 0.94. :) Our dev cluster has the
> current
> >>>  trunk on it.
> >>>
> >>>  OK let's just add these to 0.94:
> >>>  HBASE-4218
> >>>
> >>>  HBASE-4608
> >>>  HBASE-5128
> >>>  script necessary to do HBASE-5293 in 0.96
> >>>
> >>>
> >>>  And defer the rest to 0.96, and branch 0.94 soon'ish.
> >>>  Do you think you can do a 0.92 and trunk version of HBASE-5128?
> >>>
> >>>  -- Lars
> >>>
> >>>
> >>>  ________________________________
> >>>   From: Jonathan Hsieh <jon@cloudera.com>
> >>>  To: Matt Corgan <mcorgan@hotpads.com>
> >>>  Cc: lars hofhansl <lhofhansl@yahoo.com>; dev
> > <dev@hbase.apache.org>
> >>>  Sent: Friday, January 27, 2012 6:11 PM
> >>>  Subject: Re: hbase 0.94.0
> >>>
> >>>  Lars, Matt,
> >>>
> >>>  I like Matt's suggestion -- as long as it is not a burden let's
> > keep it in.
> >>>  If it becomes one, let's do what is best for the project.
> >>>
> >>>  If Cloudera needs to do the upgrade path, we'll provide one.  If it
> > doesn't
> >>>  go to the apache repo, we'll most likely open source it on github
> > or
> >>>  something, or "keep it on the jira" like some of the repair
> > ruby scripts.
> >>>
> >>>  Question -- to the Trend/SalesForce/FB folks can you talk about your
> > plans
> >>>  for upgrades?  Are you guys moving from 0.89/0.90ish to 0.92 already?
> >  or
> >>>  are you guys going to have to do the direct upgrade to 0.94 from 0.90
> > land
> >>>  as well?
> >>>
> >>>  Jon.
> >>>
> >>>
> >>>  On Fri, Jan 27, 2012 at 12:06 PM, Matt Corgan
> > <mcorgan@hotpads.com> wrote:
> >>>
> >>>  > The reason I brought it up is now that Mikhail committed the data
> > block
> >>>  > encoding I was going to take a stab at adding the prefix trie
> > encoding I
> >>>  > was working on this past summer.  My plan is to first make a
> > minimally
> >>>  > invasive patch to prove correctness.  But, after that there will
> > probably
> >>>  > be some big performance gains to be had from reworking some of the
> > things
> >>>  > like KeyValueScanner which I would not have the courage to get
> > working
> >>>  with
> >>>  > v1.
> >>>  >
> >>>  > So, that was why I asked, but all of that is still more
> > hypothetical than
> >>>  > real, and I don't even know if the first part will be done
> > before
> >>>  branching
> >>>  > .94 at the end of February.  Makes sense to me to not delete v1
> > until
> >>>  > there's a good reason to, which it doesn't look like we
> > have yet.  If I
> >>>  get
> >>>  > to the point where v1 is halting progress then we can reevaluate
> > based on
> >>>  > more specific issues.  Maybe none of the prefix trie will even
> > make
> >>>  .94...
> >>>  >
> >>>  > ..sent from my phone
> >>>  > On Jan 27, 2012 1:55 PM, "lars hofhansl"
> > <lhofhansl@yahoo.com> wrote:
> >>>  >
> >>>  >>  Hey Jon,
> >>>  >>
> >>>  >> understood. Makes 0.94 hard, though. If we decide now to have
> > a 0.90 to
> >>>  >> 0.94 upgrade path and then timing does not work out and nobody
> > signs up
> >>>  for
> >>>  >> the testing because it 0.92 is more convenient we'd have
> > gone through
> >>>  this
> >>>  >> for nothing.
> >>>  >>
> >>>  >> So... Thinking about this more I am -1 on supporting an
> > official upgrade
> >>>  >> path other then from one release to the next.
> >>>  >> That said, we do not have to break things intentionally.
> >>>  >>
> >>>  >>
> >>>  >> I'm fine pushing HBASE-2600 and HFile v1 removal out of
> > 0.94... As long
> >>>  >> as we won't havethe same argument for 0.96 :)
> >>>  >>
> >>>  >> And I am not aware of any file compatibility issues.
> >>>  >>
> >>>  >>
> >>>  >> We can also leave the 0.92 migration code in, but not
> > officially support
> >>>  >> it as 0.90 to 0.94 path.
> >>>  >> Then CDH4 can make sure (if needed) that it is all working.
> >>>  >>
> >>>  >>
> >>>  >> Does that work for you guys?
> >>>  >>
> >>>  >> -- Lars
> >>>  >>
> >>>  >>
> >>>  >>
> >>>  >> ________________________________
> >>>  >>  From: Jonathan Hsieh <jon@cloudera.com>
> >>>  >> To: dev@hbase.apache.org; lars hofhansl
> > <lhofhansl@yahoo.com>
> >>>  >> Sent: Friday, January 27, 2012 10:10 AM
> >>>  >> Subject: Re: hbase 0.94.0
> >>>  >>
> >>>  >>
> >>>  >> Lars,
> >>>  >>
> >>>  >> The upcoming CDH4 Beta HBase will be based on the latest
> > hotness,
> >>>  0.92.0.
> >>>  >> A CDH4 GA HBase will have to have an upgrade path from CDH3
> > HBase. If
> >>>  that
> >>>  >> HBase is 0.92 based, we'll test that, and if timing works
> > out and we
> >>>  decide
> >>>  >> 0.94, we'll have to have a path (0.90->0.94) for than
> > and will test
> >>>  that.
> >>>  >>
> >>>  >> HBASE-2600 is a big change of encoding of meta, while my
> > understanding
> >>>  is
> >>>  >> that 0.90->0,92 is a graceful HFile format conversion.  Are
> > there are
> >>>  >> things currently in trunk that further break compatiblity of
> > the file
> >>>  >> format? (what jiras?)
> >>>  >>
> >>>  >> Jon.
> >>>  >>
> >>>  >>
> >>>  >> On Fri, Jan 27, 2012 at 9:16 AM, lars hofhansl
> > <lhofhansl@yahoo.com>
> >>>  >> wrote:
> >>>  >>
> >>>  >> Not removing code for upgrade is fine.
> >>>  >> >
> >>>  >> >Todd, is Cloudera signing up for testing this path (0.90
> > to 0.94)?
> >>>  >> >
> >>>  >> >
> >>>  >> >Stack, what's your feeling w.r.t. to HBASE-2600, will
> > keeping the 0.90
> >>>  >> -> 0.92 migration path
> >>>  >> >make the migration code for HBASE-2600 (much) more
> > complicated in 0.94?
> >>>  >> >
> >>>  >> >
> >>>  >> >-- Lars
> >>>  >> >
> >>>  >> >
> >>>  >> >
> >>>  >> >----- Original Message -----
> >>>  >> >From: Stack <stack@duboce.net>
> >>>  >> >To: dev@hbase.apache.org
> >>>  >> >Cc:
> >>>  >> >Sent: Friday, January 27, 2012 9:02 AM
> >>>  >> >Subject: Re: hbase 0.94.0
> >>>  >> >
> >>>  >> >
> >>>  >> >On Fri, Jan 27, 2012 at 8:42 AM, Stack
> > <stack@duboce.net> wrote:
> >>>  >> >> In this particular case, there was no explicit
> > migration step needed
> >>>  >> >> going 0.90 to 0.92.  Upgrading from 0.90 to 0.94
> > should just be
> >>>  >> >> running the 0.92 to 0.94 migration script (if there
> > is one).
> >>>  >> >>
> >>>  >> >
> >>>  >> >Thinking more, the above only really holds if we keep the
> > .META.
> >>>  >> >migration code that runs in 0.92 on startup on into 0.94
> > (I have a
> >>>  >> >patch here where I'm removing it... I should put this
> > bit of code
> >>>  >> >back).
> >>>  >> >
> >>>  >> >I see Todd that you vote against removing hfile v1 in
> > 0.94.  To make
> >>>  >> >going from CDH3 to CDH4, we should not purge any migrating
> > code or old
> >>>  >> >version support.
> >>>  >> >
> >>>  >> >I'd be fine w/ that.  We'll need to work hard to
> > ensure it taking it
> >>>  >> >on as a principal for 0.94.  Ok w/ you "Iron
> > Hand" Lars?
> >>>  >> >
> >>>  >> >St.Ack
> >>>  >> >
> >>>  >> >
> >>>  >>
> >>>  >>
> >>>  >> --
> >>>  >> // Jonathan Hsieh (shay)
> >>>  >> // Software Engineer, Cloudera
> >>>  >>
> >>>  >> // jon@cloudera.com
> >>>  >>
> >>>  >
> >>>
> >>>
> >>>  --
> >>>  // Jonathan Hsieh (shay)
> >>>  // Software Engineer, Cloudera
> >>>  // jon@cloudera.com
> >>>
> >
> >
> >
> > --
> > Todd Lipcon
> > Software Engineer, Cloudera
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message