atlas-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Madhan Neethiraj (JIRA)" <>
Subject [jira] [Updated] (ATLAS-2694) Avro schema typedef and support for Avro schema evolution in Atlas
Date Fri, 25 May 2018 21:22:00 GMT


Madhan Neethiraj updated ATLAS-2694:
    Attachment: ATLAS-2694.patch

> Avro schema typedef and support for Avro schema evolution  in Atlas
> -------------------------------------------------------------------
>                 Key: ATLAS-2694
>                 URL:
>             Project: Atlas
>          Issue Type: New Feature
>          Components:  atlas-core
>            Reporter: Srikanth Venkat
>            Assignee: Barbara Eckman
>            Priority: Critical
>         Attachments: ATLAS-2694.patch, avro_atlas_types_08.json
> Currently the base types in Atlas do not include Avro schemas. It would be nice to add
typedef for Avro schema and any associated metadata to support schema evolution.
>  * For example, Avro_schema type supports:
>  ** All avro types, both primitive and complex, including union types, as fields of schema
>  ** All types have doc strings and defaults
>  ** A field of a schema can be another schema
>  ** Indefinite nesting of records, arrays.
>  ** Associated entities array attribute contains pointers to all datasets that reflect
the avro schema
>  ** Fully expanded avroNotation for use in serDe
>  ** Schema evolution features such as isLatest (Boolean) and version number
>  * Schema evolution Process
>  ** Input: avro schema
>  ** Output: new version of avro schema
>  ** Compatibility: FULL, BACKWARD, FORWARD, NONE
>  ** IsBreakingChange (Boolean): does the change produce an incompatible schema? (ie its
compatibility is not “FULL”)
>  * Atlas UI enhancement for JSON-valued attributes to support avro schema and avro schema
>  ** Currently JSON-valued attributes are fully displayed in-line with other attributes,
not pretty-printed, cluttering the display.  To support a better display, we can display
JSON-valued attributes in a one-line box that can be scrolled down, or fully expanded with
a mouse click that pretty-prints the JSON. 

This message was sent by Atlassian JIRA

View raw message