hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From lars hofhansl <lhofha...@yahoo.com>
Subject Re: hbase 0.94.0
Date Fri, 27 Jan 2012 01:30:11 GMT
Seem weird to force folks to install specific point releases to do an upgrade. But I am certainly
not opposed to have this in 0.92.1.
Do you have time to work on this script? Could probably be a ruby script to be run by the
HBase shell.



----- Original Message -----
From: Matt Corgan <mcorgan@hotpads.com>
To: dev@hbase.apache.org; lars hofhansl <lhofhansl@yahoo.com>
Cc: 
Sent: Thursday, January 26, 2012 5:21 PM
Subject: Re: hbase 0.94.0

It would be a simple, read-only script that iterates through your hbase
directory on hdfs, looks at each file, and prints out any regions or tables
that contain v1 files.  Either you've been running 0.92 for a while are are
not likely to have many v1 files, or you are using 0.92 as the "script" to
migrate your hfiles in which case all your regions will need major
compacting.  The user can copy/paste the identified regions (or whole
tables) over into the shell to major compact them.  After major compacting
all of them, run the script again to verify there are no v1 files.  Then
you are safe to upgrade from 0.92 to 0.94.

Lars - did you mean it would be too late to add a script like that to
0.92.1 release?  If it were included in the next release, then seems like
it would be safe to drop v1 support from 0.94.0.


On Thu, Jan 26, 2012 at 4:53 PM, lars hofhansl <lhofhansl@yahoo.com> wrote:

> I fear it will be close to impossible to have an upgrade path from any
> version of HBase to every future version.
> At some point we have to make the cut, or the code will littered with old
> cruft and upgrade logic, not even to speak of the testing overhead
> to verify that all old versions can be upgraded to the latest one.
>
>
> If we only support upgrade from one version to the next we can make sure
> that it is rock solid and think through all the corner cases.
>
> And then we can stop maintaining old code and focus on fixing bugs and
> adding features.
>
>
> I like Matt's idea being able to check that all HFiles did in fact upgrade
> to V2 (that falls into the "rock solid" part).
> And maybe that means it is too late to remove HFileV1 in 0.94.
>
>
>
> ----- Original Message -----
> From: Jonathan Hsieh <jon@cloudera.com>
> To: dev@hbase.apache.org
> Cc:
> Sent: Thursday, January 26, 2012 4:22 PM
> Subject: Re: hbase 0.94.0
>
> +1 to having some sort of migration mechanism especially for the files
> side. I say this out of personal interest -- I'm fairly certain that at
> some point I'm going to have to support these upgrades.
>
> Jon.
>
> On Thu, Jan 26, 2012 at 12:25 PM, Jesse Yates <jesse.k.yates@gmail.com
> >wrote:
>
> > +1 on removing it too, but maybe we should have some sort of upgrade
> > script to help make the switch?
> >
> > I'm thinking something pluggable on both sides of the upgrade, so people
> > can go from version X->Y, rather than having to upgrade first to an
> > intermediate and then to he version they want (right it would be going
> from
> > 0.20->.92->.96, IMO an excessive PITA).
> >
> > Just my two cents...
> >
> > - Jesse Yates
> >
> > Sent from my iPhone.
> >
> > On Jan 26, 2012, at 12:16 PM, lars hofhansl <lhofhansl@yahoo.com> wrote:
> >
> > > Good point.
> > > 0.94 is not branched, yet. And I think generally we do not support
> > skipping releases for upgrades.
> > > +1 on removing HFileV1 cruft for 0.94
> > >
> > >
> > > -- Lars
> > >
> > >
> > >
> > > ________________________________
> > > From: Matt Corgan <mcorgan@hotpads.com>
> > > To: dev@hbase.apache.org; Andrew Purtell <apurtell@apache.org>
> > > Sent: Thursday, January 26, 2012 11:51 AM
> > > Subject: Re: hbase 0.94.0
> > >
> > > Are there any thoughts about when it is ok to stop being backwards
> > > compatible?  Mainly thinking of HFileV1... 0.92 will convert all
> > HFileV1's
> > > to HFileV2's, so it would probably have been ok to delete the code for
> > > HFileV1 in 0.94 and force people to upgrade through 0.92.  That didn't
> > > actually happen, so it's looking like folks will be able to go straight
> > > from 0.90 to 0.94.  But, perhaps it should be deleted for 0.96?
> > >
> > >
> > > On Thu, Jan 26, 2012 at 11:41 AM, Andrew Purtell <apurtell@apache.org
> > >wrote:
> > >
> > >> Yeah, so
> > >>
> > >>     - Security (basically another coprocessor for inclusion in
> mainline,
> > >> like Constraints)
> > >>
> > >> if not in 0.92.1.
> > >>
> > >> Best regards,
> > >>
> > >>      - Andy
> > >>
> > >> Problems worthy of attack prove their worth by hitting back. - Piet
> Hein
> > >> (via Tom White)
> > >>
> > >>
> > >>> ________________________________
> > >>> From: Andrew Purtell <apurtell@apache.org>
> > >>> To: "dev@hbase.apache.org" <dev@hbase.apache.org>
> > >>> Sent: Thursday, January 26, 2012 11:28 AM
> > >>> Subject: Re: hbase 0.94.0
> > >>>
> > >>> A limited set of changes so we can get it out on that kind of
> > timeframe.
> > >> :-)
> > >>>
> > >>>   - Constraints (is ready to go, is a coprocessor, so is in the large
> > >> just a new package to drop in)
> > >>>
> > >>>   - Useful utilities for ops:
> > >>>
> > >>>      - LoadTestTool (if Ted didn't end up backporting this into
0.92)
> > >>>
> > >>>      - The store file locality thing I have mostly done, will finish
> it
> > >>>
> > >>>   - Mikhail and crew's ongoing optimizations (HBASE-4218, etc.), the
> > ones
> > >> he considers fully baked
> > >>>
> > >>> I saw wire compatibility mentioned, for 0.96 but perhaps
> > >> optional/transitional code in 0.94 as well. This is something we would
> > try
> > >> out and beat up upon in staging in earnest.
> > >>>
> > >>> Best regards,
> > >>>
> > >>>     - Andy
> > >>>
> > >>> Problems worthy of attack prove their worth by hitting back. - Piet
> > Hein
> > >> (via Tom White)
> > >>>
> > >>>
> > >>>
> > >>>> ________________________________
> > >>>> From: Stack <stack@duboce.net>
> > >>>> To: HBase Dev List <dev@hbase.apache.org>
> > >>>> Sent: Wednesday, January 25, 2012 8:34 PM
> > >>>> Subject: hbase 0.94.0
> > >>>>
> > >>>> Lets branch end of february?  No new features thereafter.  Is
this
> too
> > >>>> close in?  Would be grand if 0.94.0 shipped before hbasecon. 
What
> > >>>> should 0.94.0 have in it?  I don't mind if the list is short.
> > >>>>
> > >>>> Unless someone else wants too, I don't mind being release manager
> > >>>> (will try to run a tighter ship this time around).
> > >>>>
> > >>>> St.Ack
> > >>>>
> > >>>>
> > >>>>
> > >>>
> > >>>
> >
>
>
>
> --
> // Jonathan Hsieh (shay)
> // Software Engineer, Cloudera
> // jon@cloudera.com
>
>


Mime
View raw message