hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jeff Hammerbacher <ham...@cloudera.com>
Subject Re: Avro Gateway - "Production Quality"
Date Sun, 26 Dec 2010 20:16:16 GMT
Okay, I felt guilty about not cleaning this code up in a while.

I opened https://issues.apache.org/jira/browse/HBASE-3393 to track updating
the server to use the new join() method rather than sleep()ing for a long
time. The tests are failing for a very strange reason right now (claiming
that a field is missing on an Avro record that's clearly present in the
.java file; I'm at a loss), but I'll put my patch up in case you want to try
to make progress.

Also, Maven can't seem to find the Netty jar needed by Avro. I've got an
email out to the Avro lists to figure out what's up with that. For now, you
can just add the JBoss repository to the HBase pom.xml as directed at
http://www.jboss.org/netty/downloads.html.

I also fixed a bug where the server wasn't using the port number passed at
the command line.

I also opened https://issues.apache.org/jira/browse/HBASE-3394 to change the
build process to generate the Java code at compile time rather than forcing
us to check in the compiled Java code. That might not be preferred for some
who prefer source code to be checked in. Happy to discuss on that ticket.

Hope that helps,
Jeff

On Sun, Dec 26, 2010 at 10:50 AM, Jeff Hammerbacher <hammer@cloudera.com>wrote:

> Hey Michael,
>
> Actually, it was something more basic: AVRO-544 changed the way servers are
> started so that the gateway class could just join the Avro server thread and
> run forever, rather than having to run for a finite amount of time. I still
> need to get an HBase-side change in that uses this new feature of Avro--I
> just haven't had much time for coding of late.
>
> Other than that, I have been lax in keeping up with some of the new APIs
> introduced in 0.90 (the multi* stuff, for example). There are a few other
> cleanup tasks at https://issues.apache.org/jira/browse/HBASE-2400 that are
> worth checking out.
>
> Thanks for sending the patches my way for the Python client. I'll do my
> best to get to some of these updates soon, and would be more than happy to
> review patches for HBase, Avro, or the Python client on Github (thanks for
> the patches you've already sent my way).
>
> Regards,
> Jeff
>
>
>
> On Sun, Dec 26, 2010 at 7:16 AM, Michael Russo <mjrusso@gmail.com> wrote:
>
>> Jeff Hammerbacher stated on Quora [1] that "the Avro gateway is not
>> production quality" and made a reference to the improvements that an
>> upgrade
>> to Avro 1.4 would bring.
>>
>> I would like to understand a little better what this means.
>>
>> I reviewed the Avro changelog [2] for 1.4+, and very quickly
>> scanned o.a.h.h.avro.AvroServer.java [3] and the only Avro change that
>> really stood out in the context of significantly improving the production
>> quality of the gateway is AVRO-405 (Netty-based Java RPC server) [4].
>>
>> Is this a reasonable analysis or are there any other important elements
>> that
>> I'm missing?
>>
>> [1]:
>> http://www.quora.com/What-is-the-current-status-for-using-Avro-with-HBase
>> [2]: https://github.com/apache/avro/blob/trunk/CHANGES.txt
>> [3]:
>>
>> http://svn.apache.org/viewvc/hbase/trunk/src/main/java/org/apache/hadoop/hbase/avro/AvroServer.java?view=markup
>> [4]: https://issues.apache.org/jira/browse/AVRO-405
>>
>> Thanks,
>> Michael
>>
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message