avro-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "konstantin (JIRA)" <j...@apache.org>
Subject [jira] [Comment Edited] (AVRO-1903) Java package/class bindings for specific records
Date Fri, 18 Nov 2016 07:50:58 GMT

    [ https://issues.apache.org/jira/browse/AVRO-1903?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15676018#comment-15676018
] 

konstantin edited comment on AVRO-1903 at 11/18/16 7:50 AM:
------------------------------------------------------------

Hey Frederic,

I see little value in adding this complexity (and possible bugs and issues) to Avro.

* Java 9 will allow you to alias' imports.
* Schema's should be agnostic to language details

Regarding the patch/implementation detail, I have concerns:

* Relies on both class's generated by specific maven plugin, some people use some alternative
generation in other build tools such as sbt, as well as the runtime Avro version.
* What occurs where i have a shared generated java bundle, yet some users are on a previous
Avro 1.8.x runtime version (mainly due to other third class libraries, depending on them,
Storm, Spark, general Hadoop eco-sphere)
** Files written in Java on Avro 1.8.x (We have 1.5 PB of these on our Hadoop cluster) or
before will not contain in their datum files the schema that has this "magic" mapping/binding
as such will cause a ClassNotFound if using 1.8.new with mapping/bindings
** What occurs with compatibility with c clients, where they will write Avro files without
this special Java field. (and nor should they care this is the point of schema's being language
agnostic )
* Does this work with other languages such as Scala, Clojure that generally build/reuse java
implementations?
* There is no testing.

Also it be great to have a discussion thread on this.


was (Author: k.burstev):
Hey Frederic,

I see little value in adding this complexity (and possible bugs and issues) to Avro.

* Java 9 will allow you to alias' imports.
* Schema's should be agnostic to language details

Regarding the patch/implementation detail, I have concerns:

* Relies on both class's generated by specific maven plugin, some people use some alternative
generation in other build tools such as sbt, as well as the runtime Avro version.
* What occurs where i have a shared generated java bundle, yet some users are on a previous
Avro 1.8.x runtime version (mainly due to other third class libraries, depending on them,
Storm, Spark, general Hadoop eco-sphere)
** Files written in Java on Avro 1.8.x (We have 1.5 PB of these on our Hadoop cluster) or
before will not contain in their datum files the schema that has this "magic" mapping/binding
as such will cause a ClassNotFound if using 1.8.new with mapping/bindings
** What occurs with compatibility with c clients, where they will write Avro files without
this special Java field. (and nor should they care)
* Does this work with other languages such as Scala, Clojure that generally build/reuse java
implementations?
* There is no testing.

Also it be great to have a discussion thread on this.

> Java package/class bindings for specific records
> ------------------------------------------------
>
>                 Key: AVRO-1903
>                 URL: https://issues.apache.org/jira/browse/AVRO-1903
>             Project: Avro
>          Issue Type: New Feature
>          Components: java
>    Affects Versions: 1.8.1
>            Reporter: Frederic Boucher
>         Attachments: AVRO-1903.patch
>
>
> Naming convention of the Avro schemas we create are not always in line with the Java
coding convention. The maven-plugin to generate specific record classes should provide a way
to bind namespace or schema names to a custom Java package or class name.
> We have implemented a solution where we can add bindings into the maven-plugin configuration:
> One to one binding:
> <binding>
>     <from>com.namspace.order.state</from>
>     <to>com.package.Order</to>
> </binding>
> It will create a class "Order" into package "com.package" for schema "state" and namespace
"com.namspace.order"
> Reg-exp based binding:
> <binding>
>     <from>com.namespace.order.([a-zA-Z]+)</from>
>     <to>com.package.order.Order$1</to>
> </binding>
> Bindings are matched in order, which means the most specific bindings have to be put
before the generic ones.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message