Mailing-List: contact dev-help@hbase.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@hbase.apache.org
MIME-Version: 1.0
Sender: saint.ack@gmail.com
In-Reply-To: <CAAjhxrrYenJN+ZT5DN0RK6-39NdRwm12UWxH5jcS8ee8rz+G7Q@mail.gmail.com>
References: <CADcMMgHpEYHBUVYZfp1VhWob7B=ayC8DwTBE7bje_ZZ9Ff1-nQ@mail.gmail.com>
 <CAN+qs_NsAy-bc13xgdBwRSDoKQ-QyMOvYsrH73gP-_yCkpQeXg@mail.gmail.com>
 <CADcMMgEmkSqWZ-UPbPWBXqesK8CbZn1QwCp-y+EG=NWbSyzMPA@mail.gmail.com>
 <CAN+qs_PkFzp1Niiha_fu50d3h=c_oO9uNLG5g4OcmJPK1KoZ8w@mail.gmail.com>
 <CADcMMgFEgSE+=5oCHDJBM0+qt2XRZP0BEdQ6v-_JfSSmev3CFw@mail.gmail.com>
 <CADcMMgEsDEn1ZMAU0gEErznZ-adHNh6g+EQi1vLpZ=UpBsjnFA@mail.gmail.com>
 <CADcMMgEmiHNZVgXgXn89zFAnzTaTxXDeXg2WsweFOrM0WwVz6w@mail.gmail.com>
 <CALPvCiD_o6eBxb0uV1VgpFfumF7Zks_7D1B8sWp9YqK_qXqCmg@mail.gmail.com>
 <EC55E5AC-1702-4FD8-A3C0-B3D05FCC4F43@gmail.com> <CAAjhxrrYenJN+ZT5DN0RK6-39NdRwm12UWxH5jcS8ee8rz+G7Q@mail.gmail.com>
From: Stack <stack@duboce.net>
Date: Mon, 14 Aug 2017 11:10:46 -0700
Message-ID: <CADcMMgESxhF1LmdZCzKG7N3C2QbaoqMC_ME8LsybD566vQ7mwQ@mail.gmail.com>
Subject: Re: [DISCUSS] hbase-2.0.0 compatibility expectations
To: HBase Dev List <dev@hbase.apache.org>
Content-Type: multipart/alternative; boundary="001a1143eab0070e4b0556ba956e"
archived-at: Mon, 14 Aug 2017 18:10:57 -0000

--001a1143eab0070e4b0556ba956e
Content-Type: text/plain; charset="UTF-8"

On Fri, Aug 11, 2017 at 5:07 PM, Apekshit Sharma <appy@cloudera.com> wrote:

> Ref: deleting/deprecating some methods in HBaseAdmin
> https://github.com/apache/hbase/commit/de696cf6b653749c6bf105ef3d62d7
> a6c6923c57
> From first message:
> *" An hbase1 client can run against an hbase2 cluster but it will only be
> able to do DML (Get/Put/Scan, etc.). We do not allow being able to do admin
> ops using an hbase1 Admin client against an hbase2 cluster."*
>
> Question 1:
> These guarantees are with what version of jars?
> Does the client has to keep using corresponding 1.x jars with which it was
> compiled, OR can the 2.x jars just replace earlier ones and client is still
> expected to work correctly? (i would say no!)
>
>
Dropping hbase2 jars into an hbase1 deploy? No. No guarantee of binary
compat.

I was thinking of wire compatibility here Appy.


> Question second:
> How do we handle clients doing deprecated admin ops?
> a) make admin functions no-op by using empty method.
> b) throw exception. May/maynot crash the client - ideally shouldn't since
> methods are define with 'throws IOException'.
>

I think throwing exception the way to go. Its clean signal of change.


> c) delete the method - will crash the client
> But the question only makes sense if expect clients to work with 2.x jars.
> If not, we can just delete them, simple!
>
>
I'm not sure I follow. If a hbase1 client calls a deprecated/unimplemented
method, I was thinking it'd get an exception.


> Question 3: AMv2 related
> There are ops in old HBaseAdmin that'll be destructive for 2.0
> (closeRegion() fn).
> If clients continue to use 1.x jars and make these calls, that's not
> acceptable. We need to handle such deprecation on server sider itself.
> If only there was way of versioning the rpc service!
> Another way, but bad one is, deprecate existing ones and move logic to a
> new one. I think there are just 1-2 that need this treatment.
> Either way, need to come up with a solution!
>
>
Throw an exception pointing at the alternative API?

Thanks Appy,
S


>
> On Fri, Aug 4, 2017 at 10:08 AM, Andrew Purtell <andrew.purtell@gmail.com>
> wrote:
>
> > This is a really good question. I think many operators will give a lot of
> > leeway to data format changes as long as data can be copied from A to B
> > (perhaps with batch rewrite to upgrade (ideally, not required)) and
> > replication can be enabled to sync up to the current moment for cut over.
> >
> >
> > > On Aug 4, 2017, at 10:00 AM, Esteban Gutierrez <esteban@cloudera.com>
> > wrote:
> > >
> > > Should we add additional details around replication as well? for
> > instance,
> > > shall we consider a hbase-1.x cluster as a client for a hbase-2.x
> > cluster?
> > >
> > > Thanks for starting this discussion Stack,
> > >
> > > esteban.
> > >
> > > --
> > > Cloudera, Inc.
> > >
> > >
> > >> On Fri, Aug 4, 2017 at 1:05 AM, stack <saint.ack@gmail.com> wrote:
> > >>
> > >> Thanks Zach for clarification. Let me work up a list and then come
> back
> > to
> > >> this thread.  Jira needs an edit pass to.
> > >>
> > >> S
> > >>
> > >> On Aug 3, 2017 23:54, "Zach York" <zyork.contribution@gmail.com>
> wrote:
> > >>
> > >> This kinda helps, but these seem more like expectations. I was going
> > more
> > >> for things like HFile format changed, meta table structure changed,
> > >> coprocessor implementations changed (these are just examples, I don't
> > know
> > >> if any of these actually changed).
> > >>
> > >> More technical differences between branch-1 and branch-2 which then
> can
> > >> help us get the right expectations for compatibility.
> > >>
> > >>> On Wed, Aug 2, 2017 at 6:34 PM, Stack <stack@duboce.net> wrote:
> > >>>
> > >>> On Wed, Aug 2, 2017 at 5:25 PM, Zach York <
> > zyork.contribution@gmail.com>
> > >>> wrote:
> > >>>
> > >>>> Do we know what the major pain points for migration are? Can we
> > discuss
> > >>>> that/get a list going?
> > >>>>
> > >>>>
> > >>> Here's a few in outline:
> > >>>
> > >>> + There is issue of formats, of hbase-2.x being able to read
> hbase-1.x
> > >> data
> > >>> whether from HDFS or ZooKeeper or off the wire.
> > >>> + An hbase-1.x client should be able to Get/Put and Scan an hbase-2.x
> > >>> cluster; no holes in the API or unintelligible serializations.
> > >>> + There is then the little dance that has us rolling restart from an
> > >>> hbase-1.x cluster to hbase-2.x; i.e. upgrade master first and then it
> > >> will
> > >>> assign regions to the new hbase-2.x regionservers as they come on
> line.
> > >>> TBD.
> > >>>
> > >>> Is this what you mean sir?
> > >>>
> > >>> S
> > >>>
> > >>>
> > >>>> I think without that knowledge it is hard (for me at least :) ) to
> > >>>> determine where we should set our sights in terms of migration.
> > >>>>
> > >>>> Thanks,
> > >>>> Zach
> > >>>>
> > >>>>> On Wed, Aug 2, 2017 at 4:38 PM, Stack <stack@duboce.net> wrote:
> > >>>>>
> > >>>>> What are our expectations regards compatibility between hbase1 and
> > >>>> hbase2?
> > >>>>>
> > >>>>> Lets have a chat about it. Here are some goal posts.
> > >>>>>
> > >>>>> + You have to upgrade to hbase-1.x before you can migrate to
> hbase-2.
> > >>> No
> > >>>>> migration from < hbase-1 (Is this too onerous? Should we support
> 0.98
> > >>> =>
> > >>>>> 2.0?).
> > >>>>> + You do NOT have to upgrade to the latest release of hbase1 to
> > >> migrate
> > >>>> to
> > >>>>> hbase2; being up on hbase-1.0.0+ will be sufficient.
> > >>>>> + You'll have to update your hbase1 coprocessors to deploy them on
> > >>>> hbase2.
> > >>>>> A bunch of CP API has/will change by the time hbase2 comes out;
> e.g.
> > >>>>> watching for region split on RegionServer no longer makes sense
> given
> > >>>>> Master runs all splits now.
> > >>>>> + An hbase1 client can run against an hbase2 cluster but it will
> only
> > >>> be
> > >>>>> able to do DML (Get/Put/Scan, etc.). We do not allow being able to
> do
> > >>>> admin
> > >>>>> ops using an hbase1 Admin client against an hbase2 cluster. We have
> > >>> some
> > >>>>> egregious API violations in branch-1; e.g. we have protobuf in our
> > >> API
> > >>>> (See
> > >>>>> HBASE-15607). The notion is that we can't afford a deprecation
> cycle
> > >>>>> purging this stuff from our Admin API.
> > >>>>>
> > >>>>> What you all think?
> > >>>>>
> > >>>>> St.Ack
> > >>>>>
> > >>>>
> > >>>
> > >>
> >
>
>
>
> --
>
> -- Appy
>

--001a1143eab0070e4b0556ba956e--