hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ted Yu <yuzhih...@gmail.com>
Subject Re: A Java Client to connect to both 0.94 and 1.2.4 HBase
Date Fri, 30 Dec 2016 16:39:02 GMT
For hbase094_hbase_default.xml, currently it is
under hbase094/src/main/resources
Looks like it belongs to the test directory.

bq. use the 'Configuration#set" method to programmatically change the values

For end user, it would be more convenient to pass hbase-site.xml

Cheers

On Fri, Dec 30, 2016 at 8:19 AM, Enrico Olivelli <eolivelli@gmail.com>
wrote:

> Hi Ted,
>
> 2016-12-30 17:11 GMT+01:00 Ted Yu <yuzhihong@gmail.com>:
>
> > For scanHBase094Table():
> >
> >         baseDefaults.addResource("hbase094_hbase_default.xml");
> >
> > where hbase.rootdir points to file://
> >
>
> > How is the user supposed to plug in hbase-site.xml for the 0.94 cluster ?
> >
>
> the resource gets loaded from the classpath by the Hadoop Configuration
> API.
>
> This is just a code sample to prove that a JVM can connect to two
> incompatible versions of HBase.
>
> you can plug in your configuration by loading a "Configuration" built by
> reading your configuration file.
> Personally I do not use the .xml files for configuration of clients but use
> the 'Configuration#set" method to programmatically change the values
>
>
> >
> > For TableMigrationManager, I don't see where setBatchSize() is called.
> Does
> > this mean that batch size of 0 would be in effect ?
> >
>
> It is a bug, so the API uses its default values. See the HBase API docs
>
>
> >
> > Do you use any API specific to 1.2.4 ? Otherwise I suggest renaming
> > hbase124
> > as hbase1 since 1.x releases are compatible.
> >
>
> +1
> The only API that I use are the same as for 0.94 and 1.2
> (scan,get,put,delete.....)
>
>
> > Thanks
> >
> >
> >
> > On Fri, Dec 30, 2016 at 6:33 AM, Enrico Olivelli <eolivelli@gmail.com>
> > wrote:
> >
> > > Thanks you Duo Zhang for the interest.
> > >
> > > The example comes from a Java class which actually "migrates" data
> > > from one table in a 0.94 cluster to a 1.2.4 cluster
> > >
> > > this is the file:
> > > https://github.com/eolivelli/multi-hbase/blob/master/multi-
> > > hbase-client/src/test/java/MigrateHBaseDataExample.java
> > >
> > > this can be the foundation for a future migration tool but at this
> > > stage that project is only a proof-of-concept
> > >
> > >
> > > 2016-12-30 14:22 GMT+01:00 张铎(Duo Zhang) <palomino219@gmail.com>:
> > > > Nice try!
> > > >
> > > > Is this a migration tool? Or a thin layer on top of the original
> hbase
> > > > client to make the user be able to access both 0.94 and 1.2.4 with
> the
> > > same
> > > > API?
> > > >
> > > > Thanks.
> > > >
> > > > 2016-12-30 21:05 GMT+08:00 Enrico Olivelli <eolivelli@gmail.com>:
> > > >
> > > >> Hi all,
> > > >> The migration from 0.94 to 1.2.4 is really a pain and  some time
> ago I
> > > >> asked to this list if there was a way to connect to both a 0.94
> > > >> cluster and a 1.2.4 cluster.
> > > >>
> > > >> I have created a proof-of-concept project which is able to achieve
> my
> > > >> goal but "shading" most of the 0.94 client.
> > > >>
> > > >> This is the source code and the list of tricks to make it work:
> > > >> https://github.com/eolivelli/multi-hbase
> > > >>
> > > >> The above code simply performs a full scan from a 0.94 table and
> > > >> performs puts to a 1.2.4 cluster, but I think that most of the basic
> > > >> API of the 0.94 client will work and the full API of the 1.2.4 is
> > > >> available.
> > > >>
> > > >> I'm going to implement a more stable version of the client to be
> used
> > > >> in production.
> > > >>
> > > >> I will appreciate a lot If anyone has some time to take a look and
> > > >> give me some feedback.
> > > >>
> > > >> Cheers
> > > >> Enrico Olivelli
> > > >>
> > >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message