avro-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Pankaj Shroff <shro...@gmail.com>
Subject Re: Bypassing "handshake" in Responder
Date Wed, 27 Feb 2013 19:16:34 GMT
I guess my question is more basic - given that this is somewhat specific to
my own use case:

How does one use other forms of Encoder/Decoder implementations that are
available in the Avro library along with the Avro-Ipc SDK.

As of 1.7.3, I see that the only Encoding/Decoding that Avro-ipc supports
is the BinaryEncoding

Pankaj



On Mon, Feb 25, 2013 at 2:25 PM, Pankaj Shroff <shroffg@gmail.com> wrote:

> Doug
>
> Perhaps you answered a portion of my conundrum in another thread
> (permalink below) - but there is still the handshake and reuse of
> invocation logic question. Let me also think about this a little bit.
>
> Thanks in any case. Avro is a great tool in any case!
>
>
> http://mail-archives.apache.org/mod_mbox/avro-user/201302.mbox/%3CCALEq1Z_rt8FasjSR%2B%2BOOgE3ogrAh0Y%2BtL3z47hznuiBAtfvWmw%40mail.gmail.com%3E
>
>
> Pankaj
>
> shroffg@gmail.com
>
>
>
> On Mon, Feb 25, 2013 at 1:38 PM, Doug Cutting <cutting@apache.org> wrote:
>
>> This sounds like a different RPC wire format than Avro's.  Avro's
>> Requestor and Responder implement Avro's RPC wire format.  Avro's
>> Encode/Decoder and DatumReader/DatumWriter APIs should facilitate
>> implementation of other RPC wire formats that include Avro data.
>> Avro's Transceiver API may or may not be reusable, since it assumes
>> Avro-style framing.  Parts of Requestor and Responder *might* be
>> reusable and some refactoring of those classes *might* make such reuse
>> easier, but there's not that much logic there that's not specific to
>> Avro's wire format, so it might be just as easy to reimplement this
>> layer for a different wire format.  It's hard for me to say without
>> seeing a patch with a proposed refactoring.  Does that make sense?
>>
>> Doug
>>
>> On Mon, Feb 25, 2013 at 9:08 AM, Pankaj Shroff <shroffg@gmail.com> wrote:
>> > Hi
>> >
>> > We are using Avro for implementing an open source reference
>> implementation
>> > of the OpenRTB protocol.
>> >
>> > We have made a design goal to model the protocol using Avro protocol
>> files
>> > (.avpr) and generate types defined in the protocol schema using Avro .
>> The
>> > challenge is that the protocol does not necessarily require the use of
>> Avro/
>> > Binary wire encoding - or even the use of Avro/ RPC context. In fact
>> many
>> > counter parties have proprietary implementations supporting either
>> Protobuf
>> > or Json encoding.
>> >
>> > Now, there is a Json encoder/decoder in the Avro package but it seems
>> that
>> > the approach is a "schema-first" approach. The JsonEncoder assumes that
>> the
>> > encoding on the wire still follows the Avro Json encoding - which
>> includes a
>> > handshake followed by schema confirmation on both sides (client and
>> server).
>> >
>> > For the protocol we are implementing - this presents 2 problems if Avro/
>> > binary is not the chose encoding type for both sides - and if instead,
>> lets
>> > say, raw Json encoding is being used
>> >
>> > 1) the handshake is rather Avro specific - and we would like to
>> completely
>> > skip it if both sides have agreed on using raw json encoding - there
>> may be
>> > a separate info-exchange phase in the protocol that is protocol
>> specific and
>> > does not need to use Avro handshake. Is it possible to use Avro RPC
>> without
>> > the handshake?
>> >
>> > 2) we would like to use the data binding and schema resolution as
>> > implemented by the SpecificResponder class in Avro - but extend it to
>> use
>> > raw JSON - not Avro JSON - encodings.
>> >
>> > 3) We would prefer not to have to override the "respond(List<buffers>)"
>> > method of the base class Responder. This implementation always performs
>> > handshake and always uses BinaryEncoder/Decoder which removes any
>> > flexibility of using a different encoder /decoder in a derived class. We
>> > would prefer if the Responder or some derived class saves the chosen
>> > Decoder/ encoder as a protected property of the Responder object.
>> Instead of
>> > instantiating BinaryEncoder/ Decoder objects on the fly within the
>> respond
>> > method, it would be great if this was made more extensible and if the
>> > Encoder/Decoder can be specified during construction.
>> >
>> > 4) For future flexibility it would be great to have the handshake
>> > functionality available in sub-classes of Responder as an inherited
>> method
>> > (instead of private scope right now).
>> >
>> > I would welcome any suggestions/corrections.
>> >
>> > Pankaj
>> >
>> > --
>> > Pankaj Shroff
>> > twitter: @chompi
>> > http://github.com/chompi
>> > http://github.com/openrtb/openrtb2x
>> >
>>
>
>
>
> --
> Pankaj Shroff
> shroffG@Gmail.com
>



-- 
Pankaj Shroff
shroffG@Gmail.com

Mime
View raw message