avro-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Sean Busbey <bus...@cloudera.com>
Subject Re: Map with another map inside (unpredictable naming)
Date Mon, 27 Mar 2017 14:35:26 GMT
Your schema says that metadata is a map that has values of a few
different types, but it does not list a map type as one of them.

On Mon, Mar 27, 2017 at 6:11 AM, Dag Stockstad <dag@intercom.io> wrote:
> Hi Avro aficionados,
>
> I'm having trouble serializing a record with a nested map structure i.e. a
> map within a map. The record I'm trying to send has the following structure:
> {
>     "event_type": "some_type",
>     "data": {
>         "id": "2f720f90-ea06-4248-a72e-01eea44981ed",
>         "metadata": {
>             "some_attr": "some_value",
>             "some_map_with_unpredictable_name": {
>                 "some_attr": "some_value"
>             }
>         }
>     }
> }
>
> And the schema is this:
> {
>     "namespace": "org.example.event.avro",
>     "type": "record",
>     "name": "EventNotification",
>     "fields": [{
>         "name": "event_type",
>         "type": "string"
>     }, {
>         "name": "data",
>         "type": {
>             "type": "record",
>             "name": "EventData",
>             "fields": [{
>                 "name": "id",
>                 "type": "string"
>             }, {
>                 "name": "metadata",
>                 "type": {
>                     "type": "map",
>                     "values": [
>                         "int",
>                         "float",
>                         "string",
>                         "boolean",
>                         "long",
>                         "null"
>                     ]
>                 }
>             }]
>         }
>     }]
> }
>
> The nested map (some_map_with_unpredictable_name) is causing problems
> (serialization error). Is there any way I can have another map as a value in
> the metadata map?
>
> Due to the nature of the system, I cannot 100% predict the structure of the
> metadata field. Can Avro accomodate these requirements or do I have to fall
> back on something such as JSON for this one?
>
> Help very appreciated (I'm a bit stuck).
>
> Kind regards,
> Dag
>



-- 
busbey

Mime
View raw message