hadoop-hdfs-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Harsh J <ha...@cloudera.com>
Subject Re: HBase Avro Client
Date Wed, 19 Sep 2012 15:23:25 GMT
Hi Pankaj,

You reached the Hadoop user lists. Not everyone here may be an HBase
user. Moving this to the HBase user lists (user@hbase.apache.org).
Please use the right project's lists for best responses.

My reply inline.

On Wed, Sep 19, 2012 at 12:44 PM, Pankaj Misra
<pankaj.misra@impetus.co.in> wrote:
> Hi All,
>
> I have been searching around for the relevant information around building an avro HBase
client for the HBase Avro server running on a port(9090). I have come across the HBase Definitive
Guide, but even there I only see the the server startup part in the documentation which I
have already done.

I won't recommend using HBase Avro Server today, primarily cause it
had been marked for removal, and is removed:
https://issues.apache.org/jira/browse/HBASE-6553

Instead, try the REST or the Thrift server.

If you instead want to persist in using this anyway, read on.

> There are a few things that I do understand reading the avro and hbase documentation.
As per the documentation, HBase provides AvroServer.HBaseImpl, which is a server side of the
things. This implementation runs as a server listening to a port (9090) and acts as a bridge
between the AvroServer and HBase. The operations supported are as described in the Hbase.avpr
file.

The HBaseImpl is what you need actually. See the test-case
http://svn.apache.org/repos/asf/hbase/tags/0.94.1/src/test/java/org/apache/hadoop/hbase/avro/TestAvroServer.java
on how to implement an instance of HBaseImpl and use the various
objects such as ATableDescriptor, etc. with it. These generated
classes are already made available in the distribution.

> The doubts however I have are,
> 1. If I have to build a client that could connect to this port and could create table,
column families and insert data, what would it take?

See above.

> 2. Will I have to define a custom schema and have to generate classes out of it?

Doubt it. The docs say you do not need to do this cause for Java
things are pre-provided in a distribution.

> 3. How would I connect to the Avro server running with HBase?

Your client should just specify the right server and port address for
the Avro server.

> 4. Since the communication is rpc based, how will I specify what operation is to be performed
at the server from the client?

Via the HBaseImpl client API, same as you'd do on a regular client :)

> As I see, the documentation on this issue is very little, hence I am seeking help from
the community.
>
> Requesting for help. Thanks.
>
> Regards
> Pankaj Misra
>
> ________________________________________
> From: Pankaj Misra
> Sent: Wednesday, September 19, 2012 9:32 AM
> To: user@hadoop.apache.org
> Subject: HBase Avro Client
>
> Hi All,
>
> I have been trying to find some help around building an Avro client for HBase, but was
not able to find much help, as most of the articles were discussing stand-alone avro as a
in-memory data format and serialization engine.
>
> I was able to successfully start the avro server with
> $bin/hbase avro start
>
> The server starts up without any problems and runs on a port. Now I want to build an
avro hbase client to describe table, column families and to write data into habase using this
avro client.
>
> I intend to build this client in Java and my hbase version is 0.94.1. I could also see
java avro-tools jar version 1.7.1 in avro downloads. Do I need to use this for building my
client. Some clues around this would be very helpful. Thanks.
>
> Thanks and Regards
> Pankaj Misra
>
>
> _
>
> ________________________________
>
> Impetus Ranked in the Top 50 India’s Best Companies to Work For 2012.
>
> Impetus webcast ‘Designing a Test Automation Framework for Multi-vendor Interoperable
Systems’ available at http://lf1.me/0E/.
>
>
> NOTE: This message may contain information that is confidential, proprietary, privileged
or otherwise protected by law. The message is intended solely for the named addressee. If
received in error, please destroy and notify the sender. Any use of this email is prohibited
when received in error. Impetus does not represent, warrant and/or guarantee, that the integrity
of this communication has been maintained nor that the communication is free of errors, virus,
interception or interference.
>
> ________________________________
>
> Impetus Ranked in the Top 50 India’s Best Companies to Work For 2012.
>
> Impetus webcast ‘Designing a Test Automation Framework for Multi-vendor Interoperable
Systems’ available at http://lf1.me/0E/.
>
>
> NOTE: This message may contain information that is confidential, proprietary, privileged
or otherwise protected by law. The message is intended solely for the named addressee. If
received in error, please destroy and notify the sender. Any use of this email is prohibited
when received in error. Impetus does not represent, warrant and/or guarantee, that the integrity
of this communication has been maintained nor that the communication is free of errors, virus,
interception or interference.



-- 
Harsh J

Mime
View raw message