avro-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Bo Shi <bs1...@gmail.com>
Subject Re: Thoughts on an RPC protocol
Date Fri, 09 Apr 2010 18:56:02 GMT
On Fri, Apr 9, 2010 at 2:35 AM, Bruce Mitchener
<bruce.mitchener@gmail.com> wrote:
> Doug,
> I'm happy to hear that you like this approach!
> Allocation of channels seems to be something specific to an application.  In
> my app, I'd have a channel for the streaming data that is constantly
> arriving and a channel for making requests on and getting back answers
> immediately.  Others could have a channel per object or whatever.

One ubiquitous protocol that shares many of the same requirements and
properties (in particular multiplexed channels over a transport) is
SSH.  Their channel mechanism may provide additional inspiration:
[http://tools.ietf.org/html/rfc4254#section-5].  One interesting bit
is that SSH doesn't have an explicit channel for control commands,
instead they create additional message types for control messages that
aren't associated with any channel.  It's only a minor distinction

> Are your proxy servers custom software or are they just passing traffic
> along directly? If they're Avro-aware, then they can manage the handshaking
> process when routing to a new peer.  Is this something that is actively
> happening today or just something that is possible?
> I definitely agree about not wanting a handshake per request. For my
> application that would add a lot of overhead in terms of the data
> transmitted.  (I'm sending a lot of small requests, hopefully many thousands
> per second...)  I would be much much happier being able to have a handshake
> per connection (or per channel open).

If, as you suggest above, we enforce a 1-1 mapping of channel and avro
protocol, wouldn't that eliminate the need for a handshake on
subsequent requests on the same channel?  The handshake process could
be part of the open-channel negotiation.  I'm still wrapping my head
around the routing use-case; not sure if this meets the requirements
there though.

>  - Bruce
> On Thu, Apr 8, 2010 at 4:43 PM, Doug Cutting <cutting@apache.org> wrote:
>> Bruce,
>> Overall this looks like a good approach to me.
>> How do you anticipate allocating channels?  I'm guessing this would be one
>> per client object, that a pool of open connections to servers would be
>> maintained, and creating a new client object would allocate a new channel.
>> Currently we perform a handshake per request.  This is fairly cheap and
>> permits things like routing through proxy servers.  Different requests over
>> the same connection can talk to different backend servers running different
>> versions of the protocol.  Also consider the case where, between calls on an
>> object, the connection times out, and a new session is established and a new
>> handshake must take place.
>> That said, having a session where the handshake can be assumed vastly
>> simplifies one-way messages.  Without a response or error on which to prefix
>> a handshake response, a one-way client has no means to know that the server
>> was able to even parse its request.  Yet we'd still like a handshake for
>> one-way messages, so that clients and servers need not be versioned in
>> lockstep.  So the handshake-per-request model doesn't serve one-way messages
>> well.
>> How can we address both of these needs: to permit flexible payload routing
>> and efficient one-way messaging?
>> Doug
>> Bruce Mitchener wrote:
>>>  * Think about adding something for true one-way messages, but an empty
>>> reply frame is probably sufficient, since that still allows reporting
>>> errors
>>> if needed (or desired).

View raw message