hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jean-Daniel Cryans <jdcry...@apache.org>
Subject Re: hbase 0.94.0
Date Fri, 27 Jan 2012 01:36:06 GMT
The upgrade from 0.20 to 0.89 was kinda like that; it required having
all your regions major compacted including catalog tables.
Unfortunately the last part was hard to do since shutting down the
cluster could do a flush on META or ROOT so we added a script in the
0.20 branch (so it was never even released) to major compact an
offline cluster (it could also check if everything was major
compacted).

Having to upgrade to a point release sounds easy in comparison.

J-D

On Thu, Jan 26, 2012 at 5:30 PM, lars hofhansl <lhofhansl@yahoo.com> wrote:
> Seem weird to force folks to install specific point releases to do an upgrade. But I
am certainly not opposed to have this in 0.92.1.
> Do you have time to work on this script? Could probably be a ruby script to be run by
the HBase shell.
>
>
>
> ----- Original Message -----
> From: Matt Corgan <mcorgan@hotpads.com>
> To: dev@hbase.apache.org; lars hofhansl <lhofhansl@yahoo.com>
> Cc:
> Sent: Thursday, January 26, 2012 5:21 PM
> Subject: Re: hbase 0.94.0
>
> It would be a simple, read-only script that iterates through your hbase
> directory on hdfs, looks at each file, and prints out any regions or tables
> that contain v1 files.  Either you've been running 0.92 for a while are are
> not likely to have many v1 files, or you are using 0.92 as the "script" to
> migrate your hfiles in which case all your regions will need major
> compacting.  The user can copy/paste the identified regions (or whole
> tables) over into the shell to major compact them.  After major compacting
> all of them, run the script again to verify there are no v1 files.  Then
> you are safe to upgrade from 0.92 to 0.94.
>
> Lars - did you mean it would be too late to add a script like that to
> 0.92.1 release?  If it were included in the next release, then seems like
> it would be safe to drop v1 support from 0.94.0.
>
>
> On Thu, Jan 26, 2012 at 4:53 PM, lars hofhansl <lhofhansl@yahoo.com> wrote:
>
>> I fear it will be close to impossible to have an upgrade path from any
>> version of HBase to every future version.
>> At some point we have to make the cut, or the code will littered with old
>> cruft and upgrade logic, not even to speak of the testing overhead
>> to verify that all old versions can be upgraded to the latest one.
>>
>>
>> If we only support upgrade from one version to the next we can make sure
>> that it is rock solid and think through all the corner cases.
>>
>> And then we can stop maintaining old code and focus on fixing bugs and
>> adding features.
>>
>>
>> I like Matt's idea being able to check that all HFiles did in fact upgrade
>> to V2 (that falls into the "rock solid" part).
>> And maybe that means it is too late to remove HFileV1 in 0.94.
>>
>>
>>
>> ----- Original Message -----
>> From: Jonathan Hsieh <jon@cloudera.com>
>> To: dev@hbase.apache.org
>> Cc:
>> Sent: Thursday, January 26, 2012 4:22 PM
>> Subject: Re: hbase 0.94.0
>>
>> +1 to having some sort of migration mechanism especially for the files
>> side. I say this out of personal interest -- I'm fairly certain that at
>> some point I'm going to have to support these upgrades.
>>
>> Jon.
>>
>> On Thu, Jan 26, 2012 at 12:25 PM, Jesse Yates <jesse.k.yates@gmail.com
>> >wrote:
>>
>> > +1 on removing it too, but maybe we should have some sort of upgrade
>> > script to help make the switch?
>> >
>> > I'm thinking something pluggable on both sides of the upgrade, so people
>> > can go from version X->Y, rather than having to upgrade first to an
>> > intermediate and then to he version they want (right it would be going
>> from
>> > 0.20->.92->.96, IMO an excessive PITA).
>> >
>> > Just my two cents...
>> >
>> > - Jesse Yates
>> >
>> > Sent from my iPhone.
>> >
>> > On Jan 26, 2012, at 12:16 PM, lars hofhansl <lhofhansl@yahoo.com> wrote:
>> >
>> > > Good point.
>> > > 0.94 is not branched, yet. And I think generally we do not support
>> > skipping releases for upgrades.
>> > > +1 on removing HFileV1 cruft for 0.94
>> > >
>> > >
>> > > -- Lars
>> > >
>> > >
>> > >
>> > > ________________________________
>> > > From: Matt Corgan <mcorgan@hotpads.com>
>> > > To: dev@hbase.apache.org; Andrew Purtell <apurtell@apache.org>
>> > > Sent: Thursday, January 26, 2012 11:51 AM
>> > > Subject: Re: hbase 0.94.0
>> > >
>> > > Are there any thoughts about when it is ok to stop being backwards
>> > > compatible?  Mainly thinking of HFileV1... 0.92 will convert all
>> > HFileV1's
>> > > to HFileV2's, so it would probably have been ok to delete the code for
>> > > HFileV1 in 0.94 and force people to upgrade through 0.92.  That didn't
>> > > actually happen, so it's looking like folks will be able to go straight
>> > > from 0.90 to 0.94.  But, perhaps it should be deleted for 0.96?
>> > >
>> > >
>> > > On Thu, Jan 26, 2012 at 11:41 AM, Andrew Purtell <apurtell@apache.org
>> > >wrote:
>> > >
>> > >> Yeah, so
>> > >>
>> > >>     - Security (basically another coprocessor for inclusion in
>> mainline,
>> > >> like Constraints)
>> > >>
>> > >> if not in 0.92.1.
>> > >>
>> > >> Best regards,
>> > >>
>> > >>      - Andy
>> > >>
>> > >> Problems worthy of attack prove their worth by hitting back. - Piet
>> Hein
>> > >> (via Tom White)
>> > >>
>> > >>
>> > >>> ________________________________
>> > >>> From: Andrew Purtell <apurtell@apache.org>
>> > >>> To: "dev@hbase.apache.org" <dev@hbase.apache.org>
>> > >>> Sent: Thursday, January 26, 2012 11:28 AM
>> > >>> Subject: Re: hbase 0.94.0
>> > >>>
>> > >>> A limited set of changes so we can get it out on that kind of
>> > timeframe.
>> > >> :-)
>> > >>>
>> > >>>   - Constraints (is ready to go, is a coprocessor, so is in the
large
>> > >> just a new package to drop in)
>> > >>>
>> > >>>   - Useful utilities for ops:
>> > >>>
>> > >>>      - LoadTestTool (if Ted didn't end up backporting this into
0.92)
>> > >>>
>> > >>>      - The store file locality thing I have mostly done, will
finish
>> it
>> > >>>
>> > >>>   - Mikhail and crew's ongoing optimizations (HBASE-4218, etc.),
the
>> > ones
>> > >> he considers fully baked
>> > >>>
>> > >>> I saw wire compatibility mentioned, for 0.96 but perhaps
>> > >> optional/transitional code in 0.94 as well. This is something we would
>> > try
>> > >> out and beat up upon in staging in earnest.
>> > >>>
>> > >>> Best regards,
>> > >>>
>> > >>>     - Andy
>> > >>>
>> > >>> Problems worthy of attack prove their worth by hitting back. -
Piet
>> > Hein
>> > >> (via Tom White)
>> > >>>
>> > >>>
>> > >>>
>> > >>>> ________________________________
>> > >>>> From: Stack <stack@duboce.net>
>> > >>>> To: HBase Dev List <dev@hbase.apache.org>
>> > >>>> Sent: Wednesday, January 25, 2012 8:34 PM
>> > >>>> Subject: hbase 0.94.0
>> > >>>>
>> > >>>> Lets branch end of february?  No new features thereafter. 
Is this
>> too
>> > >>>> close in?  Would be grand if 0.94.0 shipped before hbasecon. 
What
>> > >>>> should 0.94.0 have in it?  I don't mind if the list is short.
>> > >>>>
>> > >>>> Unless someone else wants too, I don't mind being release manager
>> > >>>> (will try to run a tighter ship this time around).
>> > >>>>
>> > >>>> St.Ack
>> > >>>>
>> > >>>>
>> > >>>>
>> > >>>
>> > >>>
>> >
>>
>>
>>
>> --
>> // Jonathan Hsieh (shay)
>> // Software Engineer, Cloudera
>> // jon@cloudera.com
>>
>>
>

Mime
View raw message