avro-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From James Baldassari <jbaldass...@gmail.com>
Subject Re: Writing Unsolicited Messages to a Connected Netty Client
Date Fri, 20 Jan 2012 18:15:41 GMT
Hi Armin,

First I'd like to explain why the server-initiated messages are
problematic.  Allowing the server to send unsolicited messages back to the
client may work for some Transceiver implementations (possibly PHP), but
this change would not be compatible with NettyTransceiver.  When the
NettyTransceiver receives a message from the server, it needs to know which
callback to invoke in order to pass the message back correctly to the
client.  There could be several RPCs "in flight" concurrently, so one of
NettyTransceiver's jobs is to match up the response with the request that
initiated it.  If the client didn't initiate the RPC then NettyTransceiver
won't know where to deliver the message, unless there were some catch-all
callback that would be invoked whenever one of these "unsolicited" messages
were received.  So although you're probably only interested in the PHP
client, allowing the server to send these unsolicited messages would
potentially break NettyTransceiver (and possibly other implementations as

Shaun's idea of having the client poll the server periodically would
definitely work.  What we want to do is have the client receive
notifications from the server as they become available on the server side,
but we also don't want the client to be polling with such a high frequency
that a lot of CPU and bandwidth resources are wasted.  I think we can get
the best of both worlds by copying the Comet pattern, i.e. the "long poll"
but using the Avro RPC layer instead of (or on top of) HTTP.  First we'll
start with Shaun's update listener interface:

protocol WeatherUpdateListener {
  WeatherUpdate listenForUpdate();

The PHP client would invoke this RPC against the server in a tight loop.
On the server side, the RPC will block until there is an update that is
ready to be sent to the client.  When the client does receive an event from
the server (or some timeout occurs), the client will immediately send
another poll to the server and block until the next update is received.  In
this way the client will not be flooding the server with RPCs, but the
client will also get updates in a timely manner.

See the following for more info about Comet:


On Fri, Jan 20, 2012 at 12:44 PM, Armin Garcia <armin.garcia@arrayent.com>wrote:

> Hi Shaun,
> This is definitely another way.  I share your same concern.  I have to
> keep an eye out for high availablilty and high throughput.  I'll be
> depending on this connection to support a massive amount of data.
>                -Armin
> On Fri, Jan 20, 2012 at 9:25 AM, Shaun Williams <shaun_williams@apple.com>wrote:
>> Another solution is to use the response leg of a transaction to push
>> messages to the client, e.g. provide a server protocol like this:
>> WeatherUpdate listenForUpdate();
>> This would essentially block until an update is available.  The only
>> problem is that if the client is expecting a series of updates, it would
>> need to call this method again after receiving each update.
>> This is not an ideal solution, but it might solve your problem.
>> -Shaun
>> On Jan 20, 2012, at 8:24 AM, Armin Garcia wrote:
>> Hi James,
>> First, thank you for your response.
>> Yes, you are right.  I am trying to setup a bi-directional communication
>> link.  Your suggestion would definitely accomplish this requirement.  I was
>> hoping the same channel could be reused without having to establish another
>> uni-directional link.  Netty or rather NIO is inherently bi-directional.  I
>> am suspecting RPC by definition is only uni-directional or rather a pull
>> technology?
>> One of my goals is to support as many different language bindings using
>> Avro.  PHP is one of those languages.  Unfortunately, the PHP library can
>> only function as a client.
>>                -Armin
>> On Fri, Jan 20, 2012 at 7:47 AM, James Baldassari <jbaldassari@gmail.com>wrote:
>>> Hi Armin,
>>> Could you explain a little more about what you're trying to do?  It
>>> sounds like you want a protocol in which either client or the server
>>> initiates a remote procedure call.  The easiest way to do this is to have
>>> the client also be a server and the server also be a client.  For example,
>>> consider the following protocols:
>>> protocol WeatherClient {
>>>   double getTemperature(string postalCode);
>>>   void registerForTemperatureUpdates(string postalCode, string
>>> clientHost, int clientPort);
>>> }
>>> protocol WeatherUpdateListener {
>>>   void onTemperatureUpdate(String postalCode, double newTemperature);
>>> }
>>> The client side would use WeatherClient (and
>>> SpecificRequestor/NettyTransceiver) to request the temperature for some
>>> postal code from the server.  The client could also register with the
>>> server for temperature updates by passing the postal code it's interested
>>> in as well as the hostname/IP and port of a netty server running on the
>>> client.  The client would run its own netty server using a
>>> WeatherUpdateListener and SpecificResponder.  When the server has a
>>> temperature update to send back to the client, it would send a message to
>>> the client using the WeatherUpdateListener interface.  Is this close to
>>> what you're looking for?
>>> -James
>>> On Fri, Jan 20, 2012 at 6:34 AM, Armin Garcia <armin.garcia@arrayent.com
>>> > wrote:
>>>> I am trying to figure out how a message can be sent through a Netty
>>>> Server to a connected client.  I see the channel is stored in a group for
>>>> each client that connects to a Netty Server:
>>>> public void channelOpen(ChannelHandlerContext ctx, ChannelStateEvent e)
>>>> throws Exception
>>>> {
>>>>      e.getChannel().write(null);
>>>>      *allChannels.add(e.getChannel());*
>>>>      super.channelOpen(ctx, e);
>>>> }
>>>> The challenge is how to leverage this channel in order to send an
>>>> unsolicited message to the client.  Is there an example that writes to the
>>>> channel but not as a response to a message received?
>>>> I fully expect to take the existing Netty Server and modify it.  I
>>>> suspect the solution lies somewhere in creating a NettyDataPack then
>>>> writing it to the channel.  I'm definitely an Avro  greenhorn, so I'm a bit
>>>> unsure of how to wrangle my message into a NettyDataPack.
>>>>            -Armin

View raw message