avro-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Peter Amstutz <peter.amst...@curoverse.com>
Subject handling fields with "any" structure
Date Fri, 28 Aug 2015 13:25:36 GMT
Hello everyone,

I am using Avro to load and validate JSON documents.  Mostly this
works very well and it is straightforward to express the structure of
my document using Avro schema. However, I have a few fields which can
have "any" content.  It is impossible to declare all possible
structures in advance, and I can't use a union type of primitives
because the fields may also contain complex types (nested lists/maps)
and Avro doesn't allow named unions.

So far as I have been able to determine, this is impossible with
standard Avro schema, so I am curious if anyone else has dealt with
this problem and can suggest any workarounds.  Currently my best
(least bad) idea is to preprocess the JSON to pull out the "any"
fields and store them on the side before handing the document to Avro
for loading.  This is awkward so I would love to hear if anyone has
any other ideas.

Thanks,
Peter

Mime
View raw message