avro-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Scott Carey <scottca...@apache.org>
Subject Re: Avro Schema to SQL
Date Fri, 28 Jun 2013 16:24:59 GMT
Not all Avro schemas can be converted to SQL.  Primarily, Unions can pose
challenges, as well as recursive references.

Nested types are a mixed bag ‹ some SQL-related systems have rich support
for nested types and/or JSON (e.g. PosgtgreSQL) which can make this easier,
while others are more crude (MySQL, HIVE).

With Unions, in some cases a union field can be expanded/flattened into
multiple fields, of which only one is not null.  Recursive types can be
transformed into key references.

In general, all of these transformation strategies require decisions by the
user and potentially custom work depending on what database is involved.

Traversing an Avro Schema in Java is done via the Schema API, the Javadoc
explains it and there are many examples in the avro source code.  The type
of schema must be checked, and for each nested type a different decent into
its contained types can occur.

From:  Avinash Dongre <dongre.avinash@gmail.com>
Reply-To:  "user@avro.apache.org" <user@avro.apache.org>
Date:  Wednesday, June 19, 2013 2:31 AM
To:  "user@avro.apache.org" <user@avro.apache.org>
Subject:  Avro Schema to SQL

Is there know tool/framework available to convert Avro Schema into SQL.
If now , How Do i iterate over the schema to find out what records, enums
are there. I can think of how to achieve this with simple Schema, but I am
not able to figure out a way for nested schemas.



Thanks
Avinash




Mime
View raw message