hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Enrico Olivelli <eolive...@gmail.com>
Subject Re: A Java Client to connect to both 0.94 and 1.2.4 HBase
Date Fri, 30 Dec 2016 16:19:33 GMT
Hi Ted,

2016-12-30 17:11 GMT+01:00 Ted Yu <yuzhihong@gmail.com>:

> For scanHBase094Table():
>
>         baseDefaults.addResource("hbase094_hbase_default.xml");
>
> where hbase.rootdir points to file://
>

> How is the user supposed to plug in hbase-site.xml for the 0.94 cluster ?
>

the resource gets loaded from the classpath by the Hadoop Configuration API.

This is just a code sample to prove that a JVM can connect to two
incompatible versions of HBase.

you can plug in your configuration by loading a "Configuration" built by
reading your configuration file.
Personally I do not use the .xml files for configuration of clients but use
the 'Configuration#set" method to programmatically change the values


>
> For TableMigrationManager, I don't see where setBatchSize() is called. Does
> this mean that batch size of 0 would be in effect ?
>

It is a bug, so the API uses its default values. See the HBase API docs


>
> Do you use any API specific to 1.2.4 ? Otherwise I suggest renaming
> hbase124
> as hbase1 since 1.x releases are compatible.
>

+1
The only API that I use are the same as for 0.94 and 1.2
(scan,get,put,delete.....)


> Thanks
>
>
>
> On Fri, Dec 30, 2016 at 6:33 AM, Enrico Olivelli <eolivelli@gmail.com>
> wrote:
>
> > Thanks you Duo Zhang for the interest.
> >
> > The example comes from a Java class which actually "migrates" data
> > from one table in a 0.94 cluster to a 1.2.4 cluster
> >
> > this is the file:
> > https://github.com/eolivelli/multi-hbase/blob/master/multi-
> > hbase-client/src/test/java/MigrateHBaseDataExample.java
> >
> > this can be the foundation for a future migration tool but at this
> > stage that project is only a proof-of-concept
> >
> >
> > 2016-12-30 14:22 GMT+01:00 张铎(Duo Zhang) <palomino219@gmail.com>:
> > > Nice try!
> > >
> > > Is this a migration tool? Or a thin layer on top of the original hbase
> > > client to make the user be able to access both 0.94 and 1.2.4 with the
> > same
> > > API?
> > >
> > > Thanks.
> > >
> > > 2016-12-30 21:05 GMT+08:00 Enrico Olivelli <eolivelli@gmail.com>:
> > >
> > >> Hi all,
> > >> The migration from 0.94 to 1.2.4 is really a pain and  some time ago I
> > >> asked to this list if there was a way to connect to both a 0.94
> > >> cluster and a 1.2.4 cluster.
> > >>
> > >> I have created a proof-of-concept project which is able to achieve my
> > >> goal but "shading" most of the 0.94 client.
> > >>
> > >> This is the source code and the list of tricks to make it work:
> > >> https://github.com/eolivelli/multi-hbase
> > >>
> > >> The above code simply performs a full scan from a 0.94 table and
> > >> performs puts to a 1.2.4 cluster, but I think that most of the basic
> > >> API of the 0.94 client will work and the full API of the 1.2.4 is
> > >> available.
> > >>
> > >> I'm going to implement a more stable version of the client to be used
> > >> in production.
> > >>
> > >> I will appreciate a lot If anyone has some time to take a look and
> > >> give me some feedback.
> > >>
> > >> Cheers
> > >> Enrico Olivelli
> > >>
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message