avro-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Sébastien Launay (JIRA) <j...@apache.org>
Subject [jira] [Commented] (AVRO-1212) Protocol schema generated from reflection does not support @Union with collections
Date Fri, 30 Nov 2012 00:35:58 GMT

    [ https://issues.apache.org/jira/browse/AVRO-1212?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13506982#comment-13506982
] 

Sébastien Launay commented on AVRO-1212:
----------------------------------------

Unfortunately it's not possible to annotate the parameterized type only the whole return type,
that's why I went with this approach.
I agree this is a bit confusing and not applied like \@Nullable but we need at least to throw
an exception when parsing that configuration (\@Union on a collection/Java array).

I am wondering how to express combination of Map/Array with annotations, I was thinking of
going recursive with:
{code:java}
public interface P0 {
  @Collection(@Union{Integer.class, Long.class}) // value=... is used implicitly
    List<Object> foo();
  @Collection(@Union{Integer.class, Long.class}) // value=... is used implicitly
    Map<String, Object> bar();
  @Collection(of=@Collection(@Union{Integer.class, Long.class})) // use either value= or of=
    Map<String, Object[]> baz();
}
{code}
but recursive annotation definition is not allowed in Java today.

The {{Map}} use case is still a bit confusing because it only applies to the value type but
having {{MapOf}} and {{ArrayOf}} does not sound DRY to me (and might require checking for
avoiding misuse like using Avro type ARRAY for a Map).

Maybe only a single parameterize is allowed and for more complex use case (that looks like
a code smell though) one must create a record like:
{code:java}
public interface P0 {
  @CollectionOf(@Union{Integer.class, Long.class})
    List<Object> foo();
  @CollectionOf(@Union{Integer.class, Long.class})
    Map<String, Object> bar();
  Map<String, IntegerOrLong[]> baz();
}

public class IntegerOrLong {
  @Union{Integer.class, Long.class}
  private Object value;
  ...
}
{code}
this will requires @Union annotation to be used on fields (valid use case I think) and I guess
will not consume more bytes.

What do you think?

In the meantime, I think I found a workaround for my initial issue by using the {{@Union}}
annotation on an abstract class rather than on the method:
{code:java}
public interface P0 {
  // List<? extends Message> foo(); does not work but that's another story
  List<Message> foo(); // works
}

@Union{MessageA.class, MessageB.class}
public abstract class Message {
  ...
}
public class MessageA extends Message {
  ...
}
public class MessageB extends Message {
  ...
}
{code}

I like the {{\@AvroSchema}} though, it can be quite handy if you need full control.
                
> Protocol schema generated from reflection does not support @Union with collections
> ----------------------------------------------------------------------------------
>
>                 Key: AVRO-1212
>                 URL: https://issues.apache.org/jira/browse/AVRO-1212
>             Project: Avro
>          Issue Type: Bug
>          Components: java
>    Affects Versions: 1.7.2
>            Reporter: Sébastien Launay
>         Attachments: AVRO-1212-union-with-collections-2012-11-28.patch
>
>
> An interface using {{@Union}} for collections (Map, List, Java array) like this one:
> {code:java}
> public interface P0 {
>     @Union({String.class,Integer.class}) List<Object> foo(@Union({Integer.class,Long.class})
List<Number> l);
>   }
> {code}
> will produce the following schema where the lists have been erased by the component unions:
> {code:javascript}
> {
>   "protocol" : "P0",
>   "namespace" : "",
>   "types" : [ {
>     "type" : "record",
>     "name" : "Number",
>     "namespace" : "java.lang",
>     "fields" : [ ]
>   } ],
>   "messages" : {
>     "foo" : {
>       "request" : [ {
>         "name" : "l",
>         "type" : [ "int", "long" ]
>       } ],
>       "response" : [ "string", "int" ]
>     }
>   }
> }
> {code}
> instead of:
> {code:javascript}
> {
>   "protocol" : "P0X",
>   "namespace" : "",
>   "types" : [ {
>     "type" : "record",
>     "name" : "Object",
>     "namespace" : "java.lang",
>     "fields" : [ ]
>   }, {
>     "type" : "record",
>     "name" : "Number",
>     "namespace" : "java.lang",
>     "fields" : [ ]
>   } ],
>   "messages" : {
>     "foo" : {
>       "request" : [ {
>         "name" : "l",
>         "type" : {
>           "type" : "array",
>           "items" : [ "int", "long" ],
>           "java-class" : "java.util.List"
>         }
>       } ],
>       "response" : {
>         "type" : "array",
>         "items" : [ "string", "int" ],
>         "java-class" : "java.util.List"
>       }
>     }
>   }
> }
> {code}
> This leads to exceptions when writing a response like List<R1|R2>:
> {noformat}
> org.apache.avro.UnresolvedUnionException: Not in union [{"type":"record","name":"R1"},{"type":"record","name":"R2"}]:
[R1@19f03d7]
> 	at org.apache.avro.generic.GenericData.resolveUnion(GenericData.java:542) ~[avro-1.7.2.jar:1.7.2]
> 	at org.apache.avro.generic.GenericDatumWriter.resolveUnion(GenericDatumWriter.java:137)
~[avro-1.7.2.jar:1.7.2]
> 	at org.apache.avro.generic.GenericDatumWriter.write(GenericDatumWriter.java:70) ~[avro-1.7.2.jar:1.7.2]
> 	at org.apache.avro.reflect.ReflectDatumWriter.write(ReflectDatumWriter.java:104) ~[avro-1.7.2.jar:1.7.2]
> 	at org.apache.avro.generic.GenericDatumWriter.write(GenericDatumWriter.java:57) ~[avro-1.7.2.jar:1.7.2]
> 	at org.apache.avro.ipc.generic.GenericResponder.writeResponse(GenericResponder.java:58)
~[avro-ipc-1.7.2.jar:1.7.2]
> 	at org.apache.avro.ipc.Responder.respond(Responder.java:164) [avro-ipc-1.7.2.jar:1.7.2]
> 	at org.apache.avro.ipc.Responder.respond(Responder.java:99) [avro-ipc-1.7.2.jar:1.7.2]
> 	at org.apache.avro.ipc.ResponderServlet.doPost(ResponderServlet.java:48) [avro-ipc-1.7.2.jar:1.7.2]
> 	at javax.servlet.http.HttpServlet.service(HttpServlet.java:727) [servlet-api-2.5-20081211.jar:na]
> 	at javax.servlet.http.HttpServlet.service(HttpServlet.java:820) [servlet-api-2.5-20081211.jar:na]
> 	at org.mortbay.jetty.servlet.ServletHolder.handle(ServletHolder.java:511) [jetty-6.1.26.jar:6.1.26]
> 	at org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:401) [jetty-6.1.26.jar:6.1.26]
> 	at org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:766) [jetty-6.1.26.jar:6.1.26]
> 	at org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152) [jetty-6.1.26.jar:6.1.26]
> 	at org.mortbay.jetty.Server.handle(Server.java:322) [jetty-6.1.26.jar:6.1.26]
> 	at org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:542) [jetty-6.1.26.jar:6.1.26]
> 	at org.mortbay.jetty.HttpConnection$RequestHandler.content(HttpConnection.java:945)
[jetty-6.1.26.jar:6.1.26]
> 	at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:756) [jetty-6.1.26.jar:6.1.26]
> 	at org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:218) [jetty-6.1.26.jar:6.1.26]
> 	at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:404) [jetty-6.1.26.jar:6.1.26]
> 	at org.mortbay.io.nio.SelectChannelEndPoint.run(SelectChannelEndPoint.java:410) [jetty-6.1.26.jar:6.1.26]
> 	at org.mortbay.thread.QueuedThreadPool$PoolThread.run(QueuedThreadPool.java:582) [jetty-util-6.1.26.jar:6.1.26]
> {noformat}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message