avro-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Yang <teddyyyy...@gmail.com>
Subject Re: schema backward compatibility in RPC?
Date Tue, 31 May 2011 16:00:53 GMT
got it, thanks

On Tue, May 31, 2011 at 8:58 AM, Scott Carey <scott@richrelevance.com>wrote:

> Null is not a valid default value for string.
>
> It is a valid default value for a union of "null" and "string" however.
>
> "" is a valid default value for a String, which is what I typically use
> when I need a default string value and "" doesn't have any meaning as a
> value.
>
>
> On 5/31/11 8:53 AM, "Yang" <teddyyyy123@gmail.com> wrote:
>
> Doug:
>
> thanks a lot for the fast fix.
>
>
> it works now, but there is a minor issue, I'm not sure if this is the
> desired behavior:
>
>
> in the client schema, I set the string field "extra_field" with a default
> value of null, on the server side, the server
> probably tries to use that default value, and gave the following error:
>
>
> Tests in error:
>   testClientDiffReturnType(org.apache.avro.TestSimpleClient): Non-string
> default value for string: null
>
> Tests run: 2, Failures: 0, Errors: 1, Skipped: 0
>
>
>
> is a NULL default value for string valid? if so, then it's all fine
>
>
> Thanks
> Yang
>
> On Tue, May 31, 2011 at 3:17 AM, Doug Cutting <cutting@apache.org> wrote:
>
>> This is a bug.  I've filed an issue and attached a patch that fixes it:
>>
>> https://issues.apache.org/jira/browse/AVRO-832
>>
>> Please tell me if this fixes things for you.
>>
>> Doug
>>
>> On 05/30/2011 08:06 PM, Yang wrote:
>> > Thanks Doug.
>> >
>> > regarding "adding or removing fields from a returned record type should
>> > work",
>> > this is actually what triggered my investigation at first, and my tests
>> > seem to show that
>> > it's not the case:
>> >
>> > please see the attached schema.tar , and you can see 2 dirs,  "1/" is
>> > for server, "2/" is for client.
>> > they are created by slightly modifying the standard 1.5.1 distribution.
>> > to see the demo, you need
>> > to do  (please change the "path_to....." params below accordingly )
>> >
>> > mkdir demo
>> > cd demo
>> > tar xzf path_to_avro-1.5.1-src.tar.gz
>> > mv avro-src-1.5.1 1
>> > tar xzf path_to_avro-1.5.1-src.tar.gz
>> > mv avro-src-1.5.1 2
>> > ## the following overrides some files in standard distro
>> > tar xf path_to_schema.tar
>> > cd 1/lang/java/ipc
>> > mvn compile exec:java -Dexec.mainClass=org.apache.avro.SimpleServer
>> &>log &
>> > cd -
>> > cd 2/lang/java/ipc
>> > mvn -Dtest=org.apache.avro.TestSimpleClient  test
>> >
>> >
>> > you will see that the test testClientDiffReturnType() fails with
>> > following exception:
>> >
>> -------------------------------------------------------------------------------
>> > Test set: org.apache.avro.TestSimpleClient
>> >
>> -------------------------------------------------------------------------------
>> > Tests run: 2, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 0.392
>> > sec <<< FAILURE!
>> > testClientDiffReturnType(org.apache.avro.TestSimpleClient)  Time
>> > elapsed: 0.005 sec  <<< ERROR!
>> > java.lang.ClassCastException: org.apache.avro.test.evolution.Kind cannot
>> > be cast to java.lang.CharSequence
>> >     at org.apache.avro.test.evolution.TestRecord.put(TestRecord.java:28)
>> >     at
>> org.apache.avro.generic.GenericData.setField(GenericData.java:463)
>> >     at
>> >
>> org.apache.avro.generic.GenericDatumReader.readRecord(GenericDatumReader.java:166)
>> >     at
>> >
>> org.apache.avro.generic.GenericDatumReader.read(GenericDatumReader.java:138)
>> >     at
>> >
>> org.apache.avro.generic.GenericDatumReader.read(GenericDatumReader.java:129)
>> >     at
>> >
>> org.apache.avro.ipc.specific.SpecificRequestor.readResponse(SpecificRequestor.java:77)
>> >     at org.apache.avro.ipc.Requestor.request(Requestor.java:153)
>> >     at
>> >
>> org.apache.avro.ipc.specific.SpecificRequestor.invoke(SpecificRequestor.java:55)
>> >     at $Proxy8.echo(Unknown Source)
>> >     at
>> >
>> org.apache.avro.TestSimpleClient.testClientDiffReturnType(TestSimpleClient.java:111)
>> >     at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>> >     at
>> >
>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
>> >     at
>> >
>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>> >     at java.lang.reflect.Method.invoke(Method.java:616)
>> >     at
>> >
>> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:44)
>> >     at
>> >
>> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15)
>> >     at
>> >
>> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:41)
>> >     at
>> >
>> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:20)
>> >     at
>> >
>> org.junit.runners.BlockJUnit4ClassRunner.runNotIgnored(BlockJUnit4ClassRunner.java:79)
>> >     at
>> >
>> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:71)
>> >     at
>> >
>> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:49)
>> >     at org.junit.runners.ParentRunner$3.run(ParentRunner.java:193)
>> >     at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:52)
>> >
>> >
>> >
>> > the above error shows that avro on the client result parsing step tries
>> > to convert the old TestRecord.kind (second field) to the new
>> > TestRecord.extra_field, which is also the second field. so we get an
>> > error. so that's why I say that it seems not to be doing return schema
>> > conversion for me. maybe I need to do something to enable that?
>> >
>> > Thanks a lot
>> > Yang
>> >
>> > On Mon, May 30, 2011 at 12:36 AM, Doug Cutting <cutting@apache.org
>> > <mailto:cutting@apache.org>> wrote:
>> >
>> >     On 05/26/2011 10:10 PM, Yang wrote:
>> >     > I guess the schema resolution is done only on input params, and
>> not on
>> >     > return value?
>> >
>> >     The same resolution should be done on return values.  For example,
>> >     changing the return value from one record to another would not work
>> >     (unless you used aliases) but adding or removing fields from a
>> returned
>> >     record type should work.
>> >
>> >     > incidentally, I found that if I keep the name of an enum  type ,
>> but
>> >     > add new enum values to it (at the end of the value list), and then
>> use
>> >     > the enum type as the return
>> >     > type of my RPC call, it still works, although strictly speaking,
>> the
>> >     > enum type has already changed. but this is probably quite a
>> dangerous
>> >     > hack.
>> >
>> >     That should not be dangerous.  Avro translates enum values so that
>> >     adding & removing symbols is permitted.  As long as the reader's
>> schema
>> >     still has the symbol that the writer sends things should work.
>> >
>> >     Doug
>> >
>> >
>>
>
>

Mime
View raw message