hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Lars H <lhofha...@yahoo.com>
Subject RE: 0.92/0.94 compatibility and HBASE-5206
Date Mon, 03 Sep 2012 18:20:49 GMT
HBASE-5155 was against 0.96 only, so there was no problem.

The problem was HBASE-5206, which Ted and I did to recognize as a compatibility issue.

The fix was a good one. The issues we have now are not your fault. :)

-- Lars

"Ramkrishna.S.Vasudevan" <ramkrishna.vasudevan@huawei.com> schrieb:

>Hi 
>
>First time this issue was reported, I felt bad because HBASE-5155 was
>introduced by me.  I sincerely apologize for the mistake that I did.
>Because of my mistake I think all you guys are trying to do some
>workarounds.  Sorry for pitching in late to this thread as I was not in town
>to access mails.
>
>Next time I share surely be careful in such things.  
>
>Regards
>Ram
>
>
>
>> -----Original Message-----
>> From: Gregory Chanan [mailto:gchanan@cloudera.com]
>> Sent: Saturday, September 01, 2012 10:00 AM
>> To: dev@hbase.apache.org; lars hofhansl
>> Subject: Re: 0.92/0.94 compatibility and HBASE-5206
>> 
>> I filed HBASE-6710.
>> 
>> Greg
>> 
>> On Fri, Aug 31, 2012 at 8:21 PM, lars hofhansl <lhofhansl@yahoo.com>
>> wrote:
>> 
>> > I see. Makes sense. I made a trivial 0.94 patch for HBASE-6268.
>> > Can you file a jira for the other changes you describe below?
>> >
>> > -- Lars
>> >
>> >
>> >
>> > ________________________________
>> >  From: Gregory Chanan <gchanan@cloudera.com>
>> > To: dev@hbase.apache.org; lars hofhansl <lhofhansl@yahoo.com>
>> > Sent: Friday, August 31, 2012 7:42 PM
>> > Subject: Re: 0.92/0.94 compatibility and HBASE-5206
>> >
>> > @Lars: agreed on your point of leaving the name for the 92 znode the
>> same.
>> >
>> > On upgrading a 0.94.0 or 0.94.1 cluster out of sync, I think it would
>> work.
>> > The 0.94.2 client should work against all 0.94.2, 0.94.1, and 0.94.0
>> > versions (and all 0.92.x versions as well).  We just need to apply
>> > HBASE-6268 to 0.94.2.  So you would first upgrade the client, then
>> the
>> > server(s).  The reason this works is because the client, with HBASE-
>> 6268
>> > applied, can handle the znode being in either the 0.92 or 0.94
>> format, it
>> > doesn't need to know which is which.  I'll of course test all this
>> :).
>> >
>> > Greg
>> >
>> > On Fri, Aug 31, 2012 at 7:02 PM, lars hofhansl <lhofhansl@yahoo.com>
>> > wrote:
>> >
>> > > Also, if understand this correctly this would not help if somebody
>> had
>> > > deployed an 0.94.0 or 0.94.1 cluster and want to upgrade the client
>> and
>> > > server out of sync.
>> > > In one configuration the client would work against 0.94.0 and
>> 0.94.1 but
>> > > not against 0.94.2. In the other configuration the client would
>> work
>> > > against 0.94.2 but not against 0.94.0 or 0.94.1.
>> > >
>> > >
>> > > There is however and clean upgrade path to 0.92.2 and from there to
>> > 0.94.2
>> > > if we just fix this in 0.92.2.
>> > >
>> > >
>> > > -- Lars
>> > >
>> > >
>> > >
>> > > ________________________________
>> > >  From: lars hofhansl <lhofhansl@yahoo.com>
>> > > To: Gregory Chanan <gchanan@cloudera.com>; "dev@hbase.apache.org"
<
>> > > dev@hbase.apache.org>
>> > > Sent: Friday, August 31, 2012 6:34 PM
>> > > Subject: Re: 0.92/0.94 compatibility and HBASE-5206
>> > >
>> > > Sounds complicated. But since you are the folks with customers
>> that'll
>> > > upgrade to from 0.92 to 0.94, let's do this.
>> > >
>> > > The only input I'd have is that format we'll use going forward will
>> not
>> > > have a version attached to it.
>> > >
>> > > So maybe the 92 version would still be called
>> > > "zookeeper.znode.tableEnableDisable" and the new node could have a
>> > > different name "zookeeper.znode.tableEnableDisableNew" (or
>> something).
>> > >
>> > > -- Lars
>> > >
>> > >
>> > >
>> > > ________________________________
>> > > From: Gregory Chanan <gchanan@cloudera.com>
>> > > To: dev@hbase.apache.org; lars hofhansl <lhofhansl@yahoo.com>
>> > > Sent: Friday, August 31, 2012 5:54 PM
>> > > Subject: Re: 0.92/0.94 compatibility and HBASE-5206
>> > >
>> > >
>> > > Actually, I think we can make 0.94.2 compatible with both
>> {0.94.0,0.94.1}
>> > > and {0.92.0,0.92.1}, although one of those sets will require
>> > configuration
>> > > changes.
>> > >
>> > > The basic problem is that there is a znode for each table
>> > > "zookeeper.znode.tableEnableDisable" that is handled differently.
>> > >
>> > > On 0.92.0 and 0.92.1 the states for this table are:
>> > > [ disabled, disabling, enabling ] or deleted if the table is
>> enabled
>> > >
>> > > On 0.94.1 and 0.94.2 the states for this table are:
>> > > [ disabled, disabling, enabling, enabled ]
>> > >
>> > > What saves us is that the location of this znode is configurable.
>> So the
>> > > basic idea is to have the 0.94.2 master write two different znodes,
>> > > "zookeeper.znode.tableEnableDisabled92" and
>> > > "zookeeper.znode.tableEnableDisabled94" where the 92 node is in 92
>> > format,
>> > > the 94 node is in 94 format.  And internally, the master would only
>> use
>> > the
>> > > 94 format in order to solve the original bug HBASE-5155 solves.
>> > > We can of course make one of these the same default as exists now,
>> so we
>> > > don't need to make config changes for one of 0.92 or 0.94 clients.
>> I
>> > argue
>> > > that 0.92 clients shouldn't have to make config changes for the
>> same
>> > reason
>> > > I argued above.  But that is debatable.
>> > >
>> > > Then, I think the only question left is Stack's question of how to
>> bring
>> > > along the {0.94.0, 0.94.1} crew.  A {0.94.0, 0.94.1} client would
>> work
>> > > against a 0.94.2 cluster by just
>> > > configuring "zookeeper.znode.tableEnableDisable" in the client to
>> be
>> > > whatever "zookeeper.znode.tableEnableDisabled94" is in the cluster.
>> A
>> > > 0.94.2 client would work against both a {0.94.0, 0.94.1} and
>> {0.92.0,
>> > > 0.92.1} cluster if it had HBASE-6268 applied.  About rolling
>> upgrade from
>> > > {0.94.0, 0.94.1} to 0.94.2 -- I'd have to think about that.  Do the
>> > > regionservers ever read the tableEnableDisabled znode?
>> > >
>> > > Greg
>> > >
>> > > On Fri, Aug 31, 2012 at 4:41 PM, lars hofhansl
>> <lhofhansl@yahoo.com>
>> > > wrote:
>> > >
>> > > I guess you voiced your opinion in your initial email already (you
>> prefer
>> > > to break compatibility between 0.94.0/0.94.1 with 0.94.2).
>> > > >If that is indeed the consensus, please file a jira against
>> 0.94.2.
>> > > >
>> > > >-- Lars
>> > > >
>> > > >
>> > > >
>> > > >
>> > > >----- Original Message -----
>> > > >From: lars hofhansl <lhofhansl@yahoo.com>
>> > > >To: "dev@hbase.apache.org" <dev@hbase.apache.org>
>> > > >Cc:
>> > > >Sent: Friday, August 31, 2012 4:24 PM
>> > > >Subject: Re: 0.92/0.94 compatibility and HBASE-5206
>> > > >
>> > > >What do you think we should do?
>> > > >
>> > > >1. Breaking compatibility between minor versions is bad. (i.e. we
>> should
>> > > fix this in 0.92.2 as currently proposed)
>> > > >2. At the same time 0.92 might be in wider distribution and that
>> the
>> > > upgrade path 0.94 might be more important (and include the fix that
>> you
>> > > propose).
>> > > >
>> > > >I'm +1 on #1 and +0 on #2.
>> > > >
>> > > >
>> > > >I agree we need better cross-version integration testing and be
>> > generally
>> > > more diligent about this.
>> > > >
>> > > >-- Lars
>> > > >________________________________
>> > > >From: Gregory Chanan <gchanan@cloudera.com>
>> > > >To: dev@hbase.apache.org
>> > > >Cc: lars hofhansl <lhofhansl@yahoo.com>
>> > > >Sent: Friday, August 31, 2012 4:06 PM
>> > > >Subject: Re: 0.92/0.94 compatibility and HBASE-5206
>> > > >
>> > > >@Lars:
>> > > >you are correct that this would break compatibility between
>> {0.94.0,
>> > > >0.94.1} and 0.94.2.
>> > > >
>> > > >We clearly need better compatibility testing, these issues are
>> hard to
>> > > find
>> > > >by just looking at patches.
>> > > >
>> > > >Greg
>> > > >
>> > > >On Fri, Aug 31, 2012 at 4:03 PM, Ted Yu <yuzhihong@gmail.com>
>> wrote:
>> > > >
>> > > >> I should also apologize for HBASE-5206 where I didn't maintain
>> > > >> compatibility in the first place.
>> > > >>
>> > > >> We just need to find the solution which minimizes impact of this
>> > issue.
>> > > >>
>> > > >> Cheers
>> > > >>
>> > > >> On Fri, Aug 31, 2012 at 3:55 PM, lars hofhansl
>> <lhofhansl@yahoo.com>
>> > > >> wrote:
>> > > >>
>> > > >> > Won't we then break compatibility between 0.94.0 and 0.94.1
>> with
>> > > 0.94.2?
>> > > >> > I do not have a strong opinion about this.
>> > > >> >
>> > > >> > It was my fault that HBASE-5206 slipped into 0.94.0, I
>> apologize for
>> > > >> that.
>> > > >> >
>> > > >> > I was going to spin the first 0.94.2 today. Is the general
>> consensus
>> > > that
>> > > >> > I should wait?
>> > > >> >
>> > > >> >
>> > > >> > -- Lars
>> > > >> >
>> > > >> >
>> > > >> >
>> > > >> > ________________________________
>> > > >> >  From: Gregory Chanan <gchanan@cloudera.com>
>> > > >> > To: dev@hbase.apache.org
>> > > >> > Sent: Friday, August 31, 2012 3:13 PM
>> > > >> > Subject: 0.92/0.94 compatibility and HBASE-5206
>> > > >> >
>> > > >> > There has been some discussion on the JIRA lately about what
>> to do
>> > > about
>> > > >> a
>> > > >> > 0.92/0.94 compatibility issue.  I wanted to bring this up
to a
>> > larger
>> > > >> > audience in order to solicit additional opinions.
>> > > >> >
>> > > >> > HBASE-5206 (https://issues.apache.org/jira/browse/HBASE-5206),
>> > which
>> > > is
>> > > >> in
>> > > >> > 0.94.0 and 0.94.1, breaks compatibility with 0.92.0 and
>> 0.92.1.  The
>> > > >> issue
>> > > >> > is described in the release notes for HBASE-5155 (of which
>> > HBASE-5206
>> > > is
>> > > >> a
>> > > >> > forward port).  Excerpted here:
>> > > >> >
>> > > >> >   This issue is an incompatible change.
>> > > >> >   If an HBase client with the changes for HBASE-5155 and
a
>> server
>> > > >> (master)
>> > > >> > without the changes for HBASE-5155 is used, then the
>> is_enabled
>> > (from
>> > > >> HBase
>> > > >> > Shell) or isTableEnabled() (from HBaseAdmin) will return
false
>> > though
>> > > the
>> > > >> > table is already enabled as per the master.
>> > > >> >
>> > > >> >   If the HBase client does have the changes for HBASE-5155
and
>> the
>> > > server
>> > > >> > does not have the changes for HBASE-5155, then if we try
to
>> Enable a
>> > > >> table
>> > > >> > then the client will hang.
>> > > >> >
>> > > >> >   The reason is because,
>> > > >> >   Prior to HBASE-5155 once the table is enabled the znode
in
>> the
>> > > >> zookeeper
>> > > >> > created for the table is deleted.
>> > > >> >   After HBASE-5155 once the table is enabled the znode in
the
>> > > zookeeper
>> > > >> > created for the table is not deleted, whereas the same node
is
>> > updated
>> > > >> with
>> > > >> > the status ENABLED.
>> > > >> >
>> > > >> >   The client also expects the status of the znode in the
>> zookeeper
>> > to
>> > > be
>> > > >> in
>> > > >> > the ENABLED state if the table has been enabled successfully.
>> > > >> >   The above changes makes the client behaviour incompatible
if
>> the
>> > > client
>> > > >> > does not have this fix whereas the server has this fix.
>> > > >> >   If both the client and the server does not have this fix,
>> then the
>> > > >> > behaviour is as expected.
>> > > >> >
>> > > >> > So we have a situation where 0.92.0 and 0.92.1 clients are
not
>> > > compatible
>> > > >> > with 0.94.0 and 0.94.1 servers and visa-versa.
>> > > >> >
>> > > >> > The issue is, what should we do about the 0.94.2 server?
 As
>> far as
>> > I
>> > > can
>> > > >> > tell, we cannot make both {0.92.0,0.92.1} and {0.94.0,0.94.1}
>> > clients
>> > > >> > happy.  I propose we make the 0.94.2 server compatible with
>> the
>> > > {0.92.0,
>> > > >> > 0.92.1} client set.  My thinking is that we should optimize
>> for the
>> > > most
>> > > >> > common use case.  Given that both CDH4 and HDP 1.0 ship
>> 0.92.1, and
>> > > that
>> > > >> > 0.94 is relatively new, it seems more likely that users will
>> upgrade
>> > > >> > servers from 0.92.1 to 0.94.2+ than from an earlier 0.94
>> version.  I
>> > > >> think
>> > > >> > we can do this while actually fixing the issue that HBASE-5206
>> > > purports
>> > > >> to
>> > > >> > fix by moving the ENABLED state to a different znode.  I'll
>> even
>> > > >> volunteer
>> > > >> > to do that :).  I don't think there is an obvious or easy
>> answer
>> > > here, so
>> > > >> > wanted to get other's opinions.
>> > > >> >
>> > > >> > As for the client, HBASE-6268 (
>> > > >> > https://issues.apache.org/jira/browse/HBASE-6268) was
>> introduced in
>> > > >> 0.92.2
>> > > >> > to handle both cases of servers ({0.92.0,0.92.1} and
>> > {0.94.0,0.94.1})
>> > > by
>> > > >> > accepting both possibilities for the table znode.  I think
>> that
>> > > approach
>> > > >> > works whether we leave the server as is or go with what I've
>> > proposed
>> > > --
>> > > >> > but we should test.
>> > > >> >
>> > > >> > Greg
>> > > >>
>> > > >
>> > > >
>> > >
>> >
>
Mime
View raw message