avro-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Scott Carey <scottca...@apache.org>
Subject Re: How to add optional new record fields and/or new methods in avro-ipc?
Date Mon, 17 Oct 2011 06:29:37 GMT
On 10/13/11 5:25 AM, "常冰琳" <decstery@gmail.com> wrote:

> Hi,
> I try to use avro-ipc in my project and got a problem, if the server side
> needs to upgrade,
> add a few optional field to Request/Response object, or add a new method, thus
> change the 
> schema. 
> I use reflect, add a new String field in Request class with @Nullable in
> server side, keep client
> side unchanged, but it won't to work.
> I think protobuf supports this, just by adding new optional fields.
> So does it possible or how to support backward compatibility in other way?

> 
This should work in general.  An avro field can be promoted from a String to
a nullable String.  The inverse is not true.  A client that is expecting a
non-nullable string field response cannot read data serialized with a
nullable string.   

I am not familiar enough with the Protocol requestor or how it interacts
with the Reflect API to spot the issue.  I can't tell from the stack trace
if a resolving decoder is in use ― which is required to resolve from one
schema to another.

If you want the server side to evolve you can:
add new fields
remove fields (only if the field has a default on the client side)
promote fields as permitted (int -> long; string -> nullable string; etc.)
rename fields or other named types with aliases.

> Here is the protocol class
> 
> 
> old:
> 
> 
> 
> public interface SampleService {
>   public static class Request {
>     int id;
>     String name;
>     @Nullable
>     String address;
> 
>     public Request() {
>       this(0, "null", null);
>     }
> 
>     public Request(int id, String name, String address) {
>       this.id <http://this.id>  = id;
>       this.name <http://this.name>  = name;
>       this.address = address;
>     }
> 
>     public int getId() {
>       return id;
>     }
> 
>     public String getName() {
>       return name;
>     }
> 
>     public String getAddress() {
>       return address;
>     }
>     
>   }
>   
>   public static class Response {
>     @Nullable
>     String value;
> 
>     public Response() {
>       this("null");
>     }
>     public Response(String value) {
>       this.value = value;
>     }
> 
>     public String getValue() {
>       return value;
>     }
>   }
> 
>   int fibonacci(int n);
>   int sum(int [] nums, int base);
>   Response complexCall(Request req);
> }
> 
> 
> 
> 
> new:
> 
> 
> public interface SampleService {
>   public static class Request {
>     int id;
>     String name;
>     @Nullable
>     String address;
>     @Nullable
>     String address2;
> 
>     public Request() {
>       this(0, "null", null, null);
>     }
> 
>     public Request(int id, String name, String address, String address2) {
>       this.id <http://this.id>  = id;
>       this.name <http://this.name>  = name;
>       this.address = address;
>       this.address2 = address2;
>     }
> 
>     public int getId() {
>       return id;
>     }
> 
>     public String getName() {
>       return name;
>     }
> 
>     public String getAddress() {
>       return address;
>     }
> 
>     public String getAddress2() {
>       return address2;
>     }
>     
>   }
>   
>   public static class Response {
>     @Nullable
>     String value;
> 
>     public Response() {
>       this("null");
>     }
>     public Response(String value) {
>       this.value = value;
>     }
> 
>     public String getValue() {
>       return value;
>     }
>   }
> 
>   int fibonacci(int n);
>   int sum(int [] nums, int base);
>   Response complexCall(Request req);
> }
> 
> 
> Here is the error message:
> 
> Exception in thread "main" org.apache.avro.AvroRuntimeException:
> org.apache.avro.AvroTypeException: Found {
>   "type" : "record",
>   "name" : "Request",
>   "namespace" : "com.mycompany.app.SampleService$",
>   "fields" : [ {
>     "name" : "id",
>     "type" : "int"
>   }, {
>     "name" : "name",
>     "type" : "string"
>   }, {
>     "name" : "address",
>     "type" : [ "null", "string" ]
>   } ]
> }, expecting {
>   "type" : "record",
>   "name" : "Request",
>   "namespace" : "com.mycompany.app.SampleService$",
>   "fields" : [ {
>     "name" : "id",
>     "type" : "int"
>   }, {
>     "name" : "name",
>     "type" : "string"
>   }, {
>     "name" : "address",
>     "type" : [ "null", "string" ]
>   }, {
>     "name" : "address2",
>     "type" : [ "null", "string" ]
>   } ]
> }
> at 
> org.apache.avro.ipc.specific.SpecificRequestor.readError(SpecificRequestor.jav
> a:126)
> at org.apache.avro.ipc.Requestor$Response.getResponse(Requestor.java:555)
> at 
> org.apache.avro.ipc.Requestor$TransceiverCallback.handleResult(Requestor.java:
> 360)
> at 
> org.apache.avro.ipc.Requestor$TransceiverCallback.handleResult(Requestor.java:
> 323)
> at 
> org.apache.avro.ipc.NettyTransceiver$NettyClientAvroHandler.messageReceived(Ne
> ttyTransceiver.java:382)
> at 
> org.jboss.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChan
> nelUpstreamHandler.java:80)
> at 
> org.apache.avro.ipc.NettyTransceiver$NettyClientAvroHandler.handleUpstream(Net
> tyTransceiver.java:364)
> at 
> org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipe
> line.java:545)
> at 
> org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.se
> ndUpstream(DefaultChannelPipeline.java:754)
> at org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:302)
> at 
> org.jboss.netty.handler.codec.frame.FrameDecoder.unfoldAndFireMessageReceived(
> FrameDecoder.java:317)
> at 
> org.jboss.netty.handler.codec.frame.FrameDecoder.callDecode(FrameDecoder.java:
> 299)
> at 
> org.jboss.netty.handler.codec.frame.FrameDecoder.messageReceived(FrameDecoder.
> java:216)
> at 
> org.jboss.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChan
> nelUpstreamHandler.java:80)
> at 
> org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipe
> line.java:545)
> at 
> org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipe
> line.java:540)
> at org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:274)
> at org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:261)
> at org.jboss.netty.channel.socket.nio.NioWorker.read(NioWorker.java:349)
> at 
> org.jboss.netty.channel.socket.nio.NioWorker.processSelectedKeys(NioWorker.jav
> a:280)
> at org.jboss.netty.channel.socket.nio.NioWorker.run(NioWorker.java:200)
> at 
> org.jboss.netty.util.ThreadRenamingRunnable.run(ThreadRenamingRunnable.java:10
> 8)
> at 
> org.jboss.netty.util.internal.DeadLockProofWorker$1.run(DeadLockProofWorker.ja
> va:44)
> at 
> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java
> :886)
> at 
> 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908>
)
> at java.lang.Thread.run(Thread.java:680)
> 



Mime
View raw message