avro-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Mike Percy <mpe...@cloudera.com>
Subject Re: Avro RPC interop
Date Wed, 07 Mar 2012 23:18:59 GMT
Hi Scott & Doug,
Thank you for your responses.

I wonder if we couldn't retrofit the Netty implementation with SASL and see how close that
gets us to something that looks like it has all the desired features needed to consider standardizing
on it, or something close to it.


On Mar 6, 2012, at 8:32 AM, Doug Cutting wrote:

> A proliferation of RPC methods would create interoperability problems.
> So my idea is that Avro should specify just two standard RPC methods:
>  - HTTP; and
>  - a single alternative that permits higher-performance, secure
> implementations.
> We've yet to settle on the second.  In particular, we need to agree on
> how to implement:
> - out of order responses; and
> - authentication and encryption.
> For out-of-order responses I made a proposal in AVRO-625 that we add an
> optional call ID to request metadata.  The Netty implementation instead
> puts a call ID outside of the framing.  I can see arguments for either
> approach and don't have a strong preference.  We primarily need to
> choose one and implement it consistently.
> For authentication and encryption I have proposed SASL.  SASL's
> anonymous mechanism is easy to implement, by just placing a fixed string
> at the front of the first request and response (see the "Anonymous
> Mechanism" section of http://avro.apache.org/docs/current/sasl.html).
> With SASL, implementations can optionally negotiate authentication and
> encryption.
> I'd love to see us get this hashed out and included in 1.7.  I'd like to
> see us have at least two implementations before we declare it a standard
> part of the specification.  In Java we could add SASL support to
> NettyTransceiver and/or add out-of-order support to SaslTransceiver.
> Then we'd need someone to implement at least one other compatible
> implementation (Python?, C?).
> Doug
> On 03/04/2012 03:56 PM, Mike Percy wrote:
>> Hi Avro devs,
>> What is the latest on Avro RPC interoperability? Flume 1.x is currently using the
Avro Netty implementation as an RPC protocol, however I am not aware of compatible implementations
of this transport in languages other than Java [1] and I wonder if this protocol will ever
be compatible [2] with a future standard [3].
>> Would any of you folks recommend running RPC APIs intended for public consumption
on top of NettyServer at this time? It seems to be the one with the most active development
going on, and appears to have significant interest, which is a good sign. Since there are
other RPC transports available as well - SaslSocketServer and HttpServer - I am also wondering
what anecdotal experience others can share about these implementations, in terms of speed,
interoperability, and maturity.
>> Thanks in advance for your thoughts.
>> Regards,
>> Mike
>> [1] https://mail-archives.apache.org/mod_mbox/avro-dev/201201.mbox/%3C4F1DC9EC.1080100@apache.org%3E
>> [2] https://mail-archives.apache.org/mod_mbox/avro-dev/201201.mbox/%3C4F1A09B9.1010607@apache.org%3E
>> [3] https://issues.apache.org/jira/browse/AVRO-341 - specify avro transport in spec

View raw message