hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Francis Christopher Liu <toffer....@gmail.com>
Subject Re: [DISCUSS] hbase-2.0.0 compatibility expectations
Date Tue, 05 Sep 2017 01:55:52 GMT
Just some high-level thoughts on rolling upgradeability (I'm repeating a
few things already said).

1. No service interruption for 1.x clients while a cluster is being
ugpraded to 2.x.
  - This includes support for apis commonly used in a running system:
delegation tokens, getting region locations, etc.
2. The somewhat reverse of #1 is also necessary for Master DMLs to meta.
3. No interruption for replication between 1.x and 2.x cluster
4. Restart master first before any other service sounds reasonable to me.
Will we support RU for zkless mode?
5. 2.x should be either able to understand formats written by 1.x or online
migration is done as a preparation step.
6. Data generated by 2.x daemons should be readable in 1.x (ie HFile,
zookeeper, etc). Some deploys may have large amounts of data could be
cumbersome/impractical (ie cost, time, etc) to copy the data.

Thanks,
Francis



On Fri, Sep 1, 2017 at 10:01 PM Chia-Ping Tsai <chia7712@apache.org> wrote:

> The most of issues use hadoop flag, so the filter looks like this.
> project = HBASE AND fixVersion in (2.0.0, 2.0.0-alpha-1, 2.0.0-alpha-2,
> 2.0.0-alpha-3,
> 3.0) AND (labels in (incompatibleChange, incompatible, incompatibility) OR
> "Hadoop Flags" in ("Incompatible change"))
>
> On 2017-08-04 05:46, Ted Yu <yuzhihong@gmail.com> wrote:
> > I expanded the condition in the filter like this:
> >
> > project = HBASE AND fixVersion in (2.0.0, 2.0.0-alpha-1,  2.0.0-alpha-2,
> > 3.0) AND labels in (incompatibleChange, incompatible, incompatibility)
> >
> > Still there're only two showing up.
> >
> > On Thu, Aug 3, 2017 at 9:07 AM, Zach York <zyork.contribution@gmail.com>
> > wrote:
> >
> > > I tried to filter based on imcompatible labels and there were only two
> > > JIRAs returned [1]. I have a hard time believing that there were only
> two
> > > breaking changes from 1.x to 2.x.
> > >
> > > [1]
> > > https://issues.apache.org/jira/browse/HBASE-17957?jql=
> > > project%20%3D%20HBASE%20AND%20fixVersion%20%3D%202.0.0%
> > > 20AND%20labels%20in%20(incompatibleChange%2C%20incompatible%2C%
> > > 20incompatibility)
> > >
> > > On Thu, Aug 3, 2017 at 8:54 AM, Zach York <
> zyork.contribution@gmail.com>
> > > wrote:
> > >
> > > > This kinda helps, but these seem more like expectations. I was going
> more
> > > > for things like HFile format changed, meta table structure changed,
> > > > coprocessor implementations changed (these are just examples, I don't
> > > know
> > > > if any of these actually changed).
> > > >
> > > > More technical differences between branch-1 and branch-2 which then
> can
> > > > help us get the right expectations for compatibility.
> > > >
> > > > On Wed, Aug 2, 2017 at 6:34 PM, Stack <stack@duboce.net> wrote:
> > > >
> > > >> On Wed, Aug 2, 2017 at 5:25 PM, Zach York <
> zyork.contribution@gmail.com
> > > >
> > > >> wrote:
> > > >>
> > > >> > Do we know what the major pain points for migration are? Can
we
> > > discuss
> > > >> > that/get a list going?
> > > >> >
> > > >> >
> > > >> Here's a few in outline:
> > > >>
> > > >> + There is issue of formats, of hbase-2.x being able to read
> hbase-1.x
> > > >> data
> > > >> whether from HDFS or ZooKeeper or off the wire.
> > > >> + An hbase-1.x client should be able to Get/Put and Scan an
> hbase-2.x
> > > >> cluster; no holes in the API or unintelligible serializations.
> > > >> + There is then the little dance that has us rolling restart from
an
> > > >> hbase-1.x cluster to hbase-2.x; i.e. upgrade master first and then
> it
> > > will
> > > >> assign regions to the new hbase-2.x regionservers as they come on
> line.
> > > >> TBD.
> > > >>
> > > >> Is this what you mean sir?
> > > >>
> > > >> S
> > > >>
> > > >>
> > > >> > I think without that knowledge it is hard (for me at least :)
) to
> > > >> > determine where we should set our sights in terms of migration.
> > > >> >
> > > >> > Thanks,
> > > >> > Zach
> > > >> >
> > > >> > On Wed, Aug 2, 2017 at 4:38 PM, Stack <stack@duboce.net>
wrote:
> > > >> >
> > > >> > > What are our expectations regards compatibility between
hbase1
> and
> > > >> > hbase2?
> > > >> > >
> > > >> > > Lets have a chat about it. Here are some goal posts.
> > > >> > >
> > > >> > > + You have to upgrade to hbase-1.x before you can migrate
to
> > > hbase-2.
> > > >> No
> > > >> > > migration from < hbase-1 (Is this too onerous? Should
we support
> > > 0.98
> > > >> =>
> > > >> > > 2.0?).
> > > >> > > + You do NOT have to upgrade to the latest release of hbase1
to
> > > >> migrate
> > > >> > to
> > > >> > > hbase2; being up on hbase-1.0.0+ will be sufficient.
> > > >> > > + You'll have to update your hbase1 coprocessors to deploy
them
> on
> > > >> > hbase2.
> > > >> > > A bunch of CP API has/will change by the time hbase2 comes
out;
> e.g.
> > > >> > > watching for region split on RegionServer no longer makes
sense
> > > given
> > > >> > > Master runs all splits now.
> > > >> > > + An hbase1 client can run against an hbase2 cluster but
it will
> > > only
> > > >> be
> > > >> > > able to do DML (Get/Put/Scan, etc.). We do not allow being
able
> to
> > > do
> > > >> > admin
> > > >> > > ops using an hbase1 Admin client against an hbase2 cluster.
We
> have
> > > >> some
> > > >> > > egregious API violations in branch-1; e.g. we have protobuf
in
> our
> > > API
> > > >> > (See
> > > >> > > HBASE-15607). The notion is that we can't afford a deprecation
> cycle
> > > >> > > purging this stuff from our Admin API.
> > > >> > >
> > > >> > > What you all think?
> > > >> > >
> > > >> > > St.Ack
> > > >> > >
> > > >> >
> > > >>
> > > >
> > > >
> > >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message