avro-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Dan Schmitt <dan.schm...@gmail.com>
Subject Re: Deep polymorphic type support
Date Fri, 08 Sep 2017 18:56:26 GMT
Sounds like a job for a Union for payload.

And if the polymorphic payload is avro serialized you don't
have to know the schema ahead of time, you just need a
way to merge the new payload schema into the union as
you get it (so you can push the schema responsibility to
the client code giving you the event.)

I've been futzing with something similar on the C++ avro side
and the provided API needs some tweaks to make it go (I'm
looking at moving the fileReader/Wrtier up to work with
streams, then you can do memory array with the array stream
adapter; and then you need a way to get to addField for
the UnionSchema (which on the C++ side only has default
constructor, no way to access a Union somewhere in a schema
and have addField add the schema from the new data
without knowing a lot about the internals, so you either have
to save it when first creating the schema, and can't do it from
an existing schema, or need more code.)

     Dan S.

On Fri, Sep 8, 2017 at 2:35 PM, Greg Zoller <gzoller@gmail.com> wrote:
> Hello,
> I'm trying to learn if Avro currently supports deep polymorphic types like
> outlined here:
> https://stackoverflow.com/questions/46120578/how-might-i-represent-a-scala-trait-in-avro
> If not, could the standard be extended?
> The use case is this... I want to send an AlarmMessage, serialized as a
> Message[PayloadKind].  The receiving side would have enough type data to
> reconstruct the AlarmMessage (Alarm is of type Event, which is of type
> PayloadKind).  In other words I'd "listen" for Message[PayloadKind] and be
> handed an AlarmMessage.
> Of course we'd need to have schema entries for any kind of Event (or
> PayloadKind child for that matter), but there'd also need to be a way to
> represent these intermediate marker traits.
> Thoughts?
> Greg
> (A trivial json example of what I mean can be found here:
> https://github.com/gzoller/ScalaJack/blob/master/doc/classesAndTraits.md )

View raw message