hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jonathan Hsieh <...@cloudera.com>
Subject Re: hbase 0.94.0
Date Sat, 28 Jan 2012 02:40:07 GMT
I'm working on trunk/0.92 hbase-5128. There are things that have changed there that breaks/hangs
tests and I need to get those good before I start cluster testing.

Sent from my iPhone

On Jan 27, 2012, at 18:27, lars hofhansl <lhofhansl@yahoo.com> wrote:

> Salesforce will jumpstart with 0.94. :) Our dev cluster has the current trunk on it.
> 
> OK let's just add these to 0.94:
> HBASE-4218
> 
> HBASE-4608
> HBASE-5128
> script necessary to do HBASE-5293 in 0.96
> 
> 
> And defer the rest to 0.96, and branch 0.94 soon'ish.
> Do you think you can do a 0.92 and trunk version of HBASE-5128?
> 
> -- Lars
> 
> 
> ________________________________
> From: Jonathan Hsieh <jon@cloudera.com>
> To: Matt Corgan <mcorgan@hotpads.com> 
> Cc: lars hofhansl <lhofhansl@yahoo.com>; dev <dev@hbase.apache.org> 
> Sent: Friday, January 27, 2012 6:11 PM
> Subject: Re: hbase 0.94.0
> 
> Lars, Matt,
> 
> I like Matt's suggestion -- as long as it is not a burden let's keep it in.
> If it becomes one, let's do what is best for the project.
> 
> If Cloudera needs to do the upgrade path, we'll provide one.  If it doesn't
> go to the apache repo, we'll most likely open source it on github or
> something, or "keep it on the jira" like some of the repair ruby scripts.
> 
> Question -- to the Trend/SalesForce/FB folks can you talk about your plans
> for upgrades?  Are you guys moving from 0.89/0.90ish to 0.92 already?  or
> are you guys going to have to do the direct upgrade to 0.94 from 0.90 land
> as well?
> 
> Jon.
> 
> 
> On Fri, Jan 27, 2012 at 12:06 PM, Matt Corgan <mcorgan@hotpads.com> wrote:
> 
>> The reason I brought it up is now that Mikhail committed the data block
>> encoding I was going to take a stab at adding the prefix trie encoding I
>> was working on this past summer.  My plan is to first make a minimally
>> invasive patch to prove correctness.  But, after that there will probably
>> be some big performance gains to be had from reworking some of the things
>> like KeyValueScanner which I would not have the courage to get working with
>> v1.
>> 
>> So, that was why I asked, but all of that is still more hypothetical than
>> real, and I don't even know if the first part will be done before branching
>> .94 at the end of February.  Makes sense to me to not delete v1 until
>> there's a good reason to, which it doesn't look like we have yet.  If I get
>> to the point where v1 is halting progress then we can reevaluate based on
>> more specific issues.  Maybe none of the prefix trie will even make .94...
>> 
>> ..sent from my phone
>> On Jan 27, 2012 1:55 PM, "lars hofhansl" <lhofhansl@yahoo.com> wrote:
>> 
>>>   Hey Jon,
>>> 
>>> understood. Makes 0.94 hard, though. If we decide now to have a 0.90 to
>>> 0.94 upgrade path and then timing does not work out and nobody signs up for
>>> the testing because it 0.92 is more convenient we'd have gone through this
>>> for nothing.
>>> 
>>> So... Thinking about this more I am -1 on supporting an official upgrade
>>> path other then from one release to the next.
>>> That said, we do not have to break things intentionally.
>>> 
>>> 
>>> I'm fine pushing HBASE-2600 and HFile v1 removal out of 0.94... As long
>>> as we won't havethe same argument for 0.96 :)
>>> 
>>> And I am not aware of any file compatibility issues.
>>> 
>>> 
>>> We can also leave the 0.92 migration code in, but not officially support
>>> it as 0.90 to 0.94 path.
>>> Then CDH4 can make sure (if needed) that it is all working.
>>> 
>>> 
>>> Does that work for you guys?
>>> 
>>> -- Lars
>>> 
>>> 
>>> 
>>> ________________________________
>>>   From: Jonathan Hsieh <jon@cloudera.com>
>>> To: dev@hbase.apache.org; lars hofhansl <lhofhansl@yahoo.com>
>>> Sent: Friday, January 27, 2012 10:10 AM
>>> Subject: Re: hbase 0.94.0
>>> 
>>> 
>>> Lars,
>>> 
>>> The upcoming CDH4 Beta HBase will be based on the latest hotness, 0.92.0.
>>> A CDH4 GA HBase will have to have an upgrade path from CDH3 HBase. If that
>>> HBase is 0.92 based, we'll test that, and if timing works out and we decide
>>> 0.94, we'll have to have a path (0.90->0.94) for than and will test that.
>>> 
>>> HBASE-2600 is a big change of encoding of meta, while my understanding is
>>> that 0.90->0,92 is a graceful HFile format conversion.  Are there are
>>> things currently in trunk that further break compatiblity of the file
>>> format? (what jiras?)
>>> 
>>> Jon.
>>> 
>>> 
>>> On Fri, Jan 27, 2012 at 9:16 AM, lars hofhansl <lhofhansl@yahoo.com>
>>> wrote:
>>> 
>>> Not removing code for upgrade is fine.
>>>> 
>>>> Todd, is Cloudera signing up for testing this path (0.90 to 0.94)?
>>>> 
>>>> 
>>>> Stack, what's your feeling w.r.t. to HBASE-2600, will keeping the 0.90
>>> -> 0.92 migration path
>>>> make the migration code for HBASE-2600 (much) more complicated in 0.94?
>>>> 
>>>> 
>>>> -- Lars
>>>> 
>>>> 
>>>> 
>>>> ----- Original Message -----
>>>> From: Stack <stack@duboce.net>
>>>> To: dev@hbase.apache.org
>>>> Cc:
>>>> Sent: Friday, January 27, 2012 9:02 AM
>>>> Subject: Re: hbase 0.94.0
>>>> 
>>>> 
>>>> On Fri, Jan 27, 2012 at 8:42 AM, Stack <stack@duboce.net> wrote:
>>>>> In this particular case, there was no explicit migration step needed
>>>>> going 0.90 to 0.92.  Upgrading from 0.90 to 0.94 should just be
>>>>> running the 0.92 to 0.94 migration script (if there is one).
>>>>> 
>>>> 
>>>> Thinking more, the above only really holds if we keep the .META.
>>>> migration code that runs in 0.92 on startup on into 0.94 (I have a
>>>> patch here where I'm removing it... I should put this bit of code
>>>> back).
>>>> 
>>>> I see Todd that you vote against removing hfile v1 in 0.94.  To make
>>>> going from CDH3 to CDH4, we should not purge any migrating code or old
>>>> version support.
>>>> 
>>>> I'd be fine w/ that.  We'll need to work hard to ensure it taking it
>>>> on as a principal for 0.94.  Ok w/ you "Iron Hand" Lars?
>>>> 
>>>> St.Ack
>>>> 
>>>> 
>>> 
>>> 
>>> --
>>> // Jonathan Hsieh (shay)
>>> // Software Engineer, Cloudera
>>> 
>>> // jon@cloudera.com
>>> 
>> 
> 
> 
> -- 
> // Jonathan Hsieh (shay)
> // Software Engineer, Cloudera
> // jon@cloudera.com

Mime
View raw message