avro-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Martin Kleppmann (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (AVRO-695) Cycle Reference Support
Date Thu, 03 Jul 2014 15:42:25 GMT

    [ https://issues.apache.org/jira/browse/AVRO-695?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14051597#comment-14051597

Martin Kleppmann commented on AVRO-695:

I don't mean to be grouchy, but I'm not convinced this is a feature that Avro really needs
to have. It adds complexity to all the language implementations of Avro that need to support
it, and it creates difficulties for anyone who needs to map between Avro's data model and
another data model (e.g. representing Avro data in JSON format, or accessing Avro data in
Pig through Pig's type system).

For an application that needs to model an arbitrary graph of references between objects in
Avro, I would argue that it's actually better for the object IDs to be explicitly visible
in the data model, rather than being automatically resolved and thus hidden.

If there's some really awesome use case for circular references that I'm missing, I'm open
to being convinced. But I don't currently see it.

> Cycle Reference Support
> -----------------------
>                 Key: AVRO-695
>                 URL: https://issues.apache.org/jira/browse/AVRO-695
>             Project: Avro
>          Issue Type: New Feature
>          Components: spec
>    Affects Versions: 1.7.6
>            Reporter: Moustapha Cherri
>         Attachments: avro-1.4.1-cycle.patch.gz, avro-1.4.1-cycle.patch.gz, avro_circular_references.zip,
avro_circular_refs_2014_06_14.zip, circular_refs_and_nonstring_map_keys_2014_06_25.zip
>   Original Estimate: 672h
>  Remaining Estimate: 672h
> This is a proposed implementation to add cycle reference support to Avro. It basically
introduce a new type named Cycle. Cycles contains a string representing the path to the other
> For example if we have an object of type Message that have a member named previous with
type Message too. If we have have this hierarchy:
> message
>   previous : message2
> message2
>   previous : message2
> When serializing the cycle path for "message2.previous" will be "previous".
> The implementation depend on ANTLR to evaluate those cycle at read time to resolve them.
I used ANTLR 3.2. This dependency is not mandated; I just used ANTLR to speed thing up. I
kept in this implementation the generated code from ANTLR though this should not be the case
as this should be generated during the build. I only updated the Java code.
> I did not make full unit testing but you can find "avrotest.Main" class that can be used
a preliminary test.
> Please do not hesitate to contact me for further clarification if this seems interresting.
> Best regards,
> Moustapha Cherri

This message was sent by Atlassian JIRA

View raw message