avro-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Zoltan Farkas (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (AVRO-1723) Add support for forward declarations in avro IDL
Date Sun, 30 Apr 2017 20:56:04 GMT

    [ https://issues.apache.org/jira/browse/AVRO-1723?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15990376#comment-15990376

Zoltan Farkas commented on AVRO-1723:

[~thiru_mg] Thanks for review.

your cleanup + fixes looks good.

Regarding the order of definition of named schema elements, it will not matter anymore.

This implementation will resolve all named elements not only records.

I just tested the following:

   /* Name Value record */
   record ANameValue {
      /** the name */
      string name;
      /** the value */
      string value;
      /* is the value a json object */
      ValueType type = "PLAIN";
   enum ValueType {JSON, BASE64BIN, PLAIN}

while this is not supported with AVRO without this change.
It will work just fine from now on. 
Worthwhile adding a unit test.

What we should also do add a jira to move the generic schema walker we created for this into
the core avro lib and see if there are other places this can be re-used as [~rdblue] suggested:

"Also, you might be able to simplify what's happening here by using the visitor pattern. I've
been thinking that it would be helpful to have an internal SchemaVisitor for this kind of
operation. That would cut down on the size of these methods and separate the recursion from
the logic you're trying to implement. This may be a good time to add it to the core library."

This is the reason of existence of the CloningVisitor, to add one other use case for this
generic schema walker... 

thank you!

> Add support for forward declarations in avro IDL
> ------------------------------------------------
>                 Key: AVRO-1723
>                 URL: https://issues.apache.org/jira/browse/AVRO-1723
>             Project: Avro
>          Issue Type: Improvement
>    Affects Versions: 1.8.0
>            Reporter: Zoltan Farkas
>            Assignee: Zoltan Farkas
>         Attachments: AVRO-1723.patch
> Currently Recursive data structures like:
> record SampleNode {
>    int count = 0;
>    array<SamplePair> samples = [];
> }
> record SamplePair {
>  string name;
>  SampleNode node;
> }
> It is not possible to declare in IDL,
> however it is possible to declare in avsc (with fix from https://issues.apache.org/jira/browse/AVRO-1667
> It is actually not complicated to implement, here is some detail on a possible implementation:
> https://github.com/zolyfarkas/avro/commit/210c50105717149f3daa39b8d4160b8548b8e363
> This would close a capability gap with google protocol buffers...

This message was sent by Atlassian JIRA

View raw message