hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Andrew Purtell <apurt...@apache.org>
Subject Re: hbase 0.94.0
Date Mon, 30 Jan 2012 19:40:27 GMT
> Maybe the discussion should be whether 0.94 should be a time-gated or
> feature-gated release? IMO we already have enough good stuff in there
> on the perf front. It will be great if the checksum improvement makes
> it, but if it doesn't, I'd rather have a release than delay for it.


Time gated releases makes sense going forward for a few reasons, mark me down for that option.

 
Best regards,

    - Andy

Problems worthy of attack prove their worth by hitting back. - Piet Hein (via Tom White) 


----- Original Message -----
> From: Todd Lipcon <todd@cloudera.com>
> To: dev@hbase.apache.org
> Cc: lars hofhansl <lhofhansl@yahoo.com>; Matt Corgan <mcorgan@hotpads.com>
> Sent: Monday, January 30, 2012 10:44 AM
> Subject: Re: hbase 0.94.0
> 
> On Sun, Jan 29, 2012 at 7:30 AM, Ted Yu <yuzhihong@gmail.com> wrote:
>>  Initial results from HBASE-5074, support checksums in HBase block cache,
>>  look promising.
>> 
>>  Shall we discuss whether 0.94 should include it (assuming Dhruba can finish
>>  the feature in Feb.) ?
> 
> If it's a time-based release, then there's nothing to discuss. Either
> it's done in time, in which case it's part of 94, or it's not, in
> which case it's part of 96, right?
> 
> Maybe the discussion should be whether 0.94 should be a time-gated or
> feature-gated release? IMO we already have enough good stuff in there
> on the perf front. It will be great if the checksum improvement makes
> it, but if it doesn't, I'd rather have a release than delay for it.
> 
> -Todd
> 
>> 
>>  Cheers
>> 
>>  On Fri, Jan 27, 2012 at 6:27 PM, lars hofhansl <lhofhansl@yahoo.com> 
> wrote:
>> 
>>>  Salesforce will jumpstart with 0.94. :) Our dev cluster has the current
>>>  trunk on it.
>>> 
>>>  OK let's just add these to 0.94:
>>>  HBASE-4218
>>> 
>>>  HBASE-4608
>>>  HBASE-5128
>>>  script necessary to do HBASE-5293 in 0.96
>>> 
>>> 
>>>  And defer the rest to 0.96, and branch 0.94 soon'ish.
>>>  Do you think you can do a 0.92 and trunk version of HBASE-5128?
>>> 
>>>  -- Lars
>>> 
>>> 
>>>  ________________________________
>>>   From: Jonathan Hsieh <jon@cloudera.com>
>>>  To: Matt Corgan <mcorgan@hotpads.com>
>>>  Cc: lars hofhansl <lhofhansl@yahoo.com>; dev 
> <dev@hbase.apache.org>
>>>  Sent: Friday, January 27, 2012 6:11 PM
>>>  Subject: Re: hbase 0.94.0
>>> 
>>>  Lars, Matt,
>>> 
>>>  I like Matt's suggestion -- as long as it is not a burden let's 
> keep it in.
>>>  If it becomes one, let's do what is best for the project.
>>> 
>>>  If Cloudera needs to do the upgrade path, we'll provide one.  If it 
> doesn't
>>>  go to the apache repo, we'll most likely open source it on github 
> or
>>>  something, or "keep it on the jira" like some of the repair 
> ruby scripts.
>>> 
>>>  Question -- to the Trend/SalesForce/FB folks can you talk about your 
> plans
>>>  for upgrades?  Are you guys moving from 0.89/0.90ish to 0.92 already? 
>  or
>>>  are you guys going to have to do the direct upgrade to 0.94 from 0.90 
> land
>>>  as well?
>>> 
>>>  Jon.
>>> 
>>> 
>>>  On Fri, Jan 27, 2012 at 12:06 PM, Matt Corgan 
> <mcorgan@hotpads.com> wrote:
>>> 
>>>  > The reason I brought it up is now that Mikhail committed the data 
> block
>>>  > encoding I was going to take a stab at adding the prefix trie 
> encoding I
>>>  > was working on this past summer.  My plan is to first make a 
> minimally
>>>  > invasive patch to prove correctness.  But, after that there will 
> probably
>>>  > be some big performance gains to be had from reworking some of the 
> things
>>>  > like KeyValueScanner which I would not have the courage to get 
> working
>>>  with
>>>  > v1.
>>>  >
>>>  > So, that was why I asked, but all of that is still more 
> hypothetical than
>>>  > real, and I don't even know if the first part will be done 
> before
>>>  branching
>>>  > .94 at the end of February.  Makes sense to me to not delete v1 
> until
>>>  > there's a good reason to, which it doesn't look like we 
> have yet.  If I
>>>  get
>>>  > to the point where v1 is halting progress then we can reevaluate 
> based on
>>>  > more specific issues.  Maybe none of the prefix trie will even 
> make
>>>  .94...
>>>  >
>>>  > ..sent from my phone
>>>  > On Jan 27, 2012 1:55 PM, "lars hofhansl" 
> <lhofhansl@yahoo.com> wrote:
>>>  >
>>>  >>  Hey Jon,
>>>  >>
>>>  >> understood. Makes 0.94 hard, though. If we decide now to have 
> a 0.90 to
>>>  >> 0.94 upgrade path and then timing does not work out and nobody 
> signs up
>>>  for
>>>  >> the testing because it 0.92 is more convenient we'd have 
> gone through
>>>  this
>>>  >> for nothing.
>>>  >>
>>>  >> So... Thinking about this more I am -1 on supporting an 
> official upgrade
>>>  >> path other then from one release to the next.
>>>  >> That said, we do not have to break things intentionally.
>>>  >>
>>>  >>
>>>  >> I'm fine pushing HBASE-2600 and HFile v1 removal out of 
> 0.94... As long
>>>  >> as we won't havethe same argument for 0.96 :)
>>>  >>
>>>  >> And I am not aware of any file compatibility issues.
>>>  >>
>>>  >>
>>>  >> We can also leave the 0.92 migration code in, but not 
> officially support
>>>  >> it as 0.90 to 0.94 path.
>>>  >> Then CDH4 can make sure (if needed) that it is all working.
>>>  >>
>>>  >>
>>>  >> Does that work for you guys?
>>>  >>
>>>  >> -- Lars
>>>  >>
>>>  >>
>>>  >>
>>>  >> ________________________________
>>>  >>  From: Jonathan Hsieh <jon@cloudera.com>
>>>  >> To: dev@hbase.apache.org; lars hofhansl 
> <lhofhansl@yahoo.com>
>>>  >> Sent: Friday, January 27, 2012 10:10 AM
>>>  >> Subject: Re: hbase 0.94.0
>>>  >>
>>>  >>
>>>  >> Lars,
>>>  >>
>>>  >> The upcoming CDH4 Beta HBase will be based on the latest 
> hotness,
>>>  0.92.0.
>>>  >> A CDH4 GA HBase will have to have an upgrade path from CDH3 
> HBase. If
>>>  that
>>>  >> HBase is 0.92 based, we'll test that, and if timing works 
> out and we
>>>  decide
>>>  >> 0.94, we'll have to have a path (0.90->0.94) for than 
> and will test
>>>  that.
>>>  >>
>>>  >> HBASE-2600 is a big change of encoding of meta, while my 
> understanding
>>>  is
>>>  >> that 0.90->0,92 is a graceful HFile format conversion.  Are 
> there are
>>>  >> things currently in trunk that further break compatiblity of 
> the file
>>>  >> format? (what jiras?)
>>>  >>
>>>  >> Jon.
>>>  >>
>>>  >>
>>>  >> On Fri, Jan 27, 2012 at 9:16 AM, lars hofhansl 
> <lhofhansl@yahoo.com>
>>>  >> wrote:
>>>  >>
>>>  >> Not removing code for upgrade is fine.
>>>  >> >
>>>  >> >Todd, is Cloudera signing up for testing this path (0.90 
> to 0.94)?
>>>  >> >
>>>  >> >
>>>  >> >Stack, what's your feeling w.r.t. to HBASE-2600, will 
> keeping the 0.90
>>>  >> -> 0.92 migration path
>>>  >> >make the migration code for HBASE-2600 (much) more 
> complicated in 0.94?
>>>  >> >
>>>  >> >
>>>  >> >-- Lars
>>>  >> >
>>>  >> >
>>>  >> >
>>>  >> >----- Original Message -----
>>>  >> >From: Stack <stack@duboce.net>
>>>  >> >To: dev@hbase.apache.org
>>>  >> >Cc:
>>>  >> >Sent: Friday, January 27, 2012 9:02 AM
>>>  >> >Subject: Re: hbase 0.94.0
>>>  >> >
>>>  >> >
>>>  >> >On Fri, Jan 27, 2012 at 8:42 AM, Stack 
> <stack@duboce.net> wrote:
>>>  >> >> In this particular case, there was no explicit 
> migration step needed
>>>  >> >> going 0.90 to 0.92.  Upgrading from 0.90 to 0.94 
> should just be
>>>  >> >> running the 0.92 to 0.94 migration script (if there 
> is one).
>>>  >> >>
>>>  >> >
>>>  >> >Thinking more, the above only really holds if we keep the 
> .META.
>>>  >> >migration code that runs in 0.92 on startup on into 0.94 
> (I have a
>>>  >> >patch here where I'm removing it... I should put this 
> bit of code
>>>  >> >back).
>>>  >> >
>>>  >> >I see Todd that you vote against removing hfile v1 in 
> 0.94.  To make
>>>  >> >going from CDH3 to CDH4, we should not purge any migrating 
> code or old
>>>  >> >version support.
>>>  >> >
>>>  >> >I'd be fine w/ that.  We'll need to work hard to 
> ensure it taking it
>>>  >> >on as a principal for 0.94.  Ok w/ you "Iron 
> Hand" Lars?
>>>  >> >
>>>  >> >St.Ack
>>>  >> >
>>>  >> >
>>>  >>
>>>  >>
>>>  >> --
>>>  >> // Jonathan Hsieh (shay)
>>>  >> // Software Engineer, Cloudera
>>>  >>
>>>  >> // jon@cloudera.com
>>>  >>
>>>  >
>>> 
>>> 
>>>  --
>>>  // Jonathan Hsieh (shay)
>>>  // Software Engineer, Cloudera
>>>  // jon@cloudera.com
>>> 
> 
> 
> 
> -- 
> Todd Lipcon
> Software Engineer, Cloudera
> 

Mime
View raw message