Return-Path: 
 <user-return-2739-apmail-avro-user-archive=avro.apache.org@avro.apache.org>
X-Original-To: apmail-avro-user-archive@www.apache.org
Delivered-To: apmail-avro-user-archive@www.apache.org
Received: from mail.apache.org (hermes.apache.org [140.211.11.3])
	by minotaur.apache.org (Postfix) with SMTP id A4D8610907
	for <apmail-avro-user-archive@www.apache.org>;
 Tue, 15 Oct 2013 23:50:30 +0000 (UTC)
Received: (qmail 72946 invoked by uid 500); 15 Oct 2013 23:50:30 -0000
Delivered-To: apmail-avro-user-archive@avro.apache.org
Received: (qmail 72902 invoked by uid 500); 15 Oct 2013 23:50:30 -0000
Mailing-List: contact user-help@avro.apache.org; run by ezmlm
Precedence: bulk
List-Help: <mailto:user-help@avro.apache.org>
List-Unsubscribe: <mailto:user-unsubscribe@avro.apache.org>
List-Post: <mailto:user@avro.apache.org>
List-Id: <user.avro.apache.org>
Reply-To: user@avro.apache.org
Delivered-To: mailing list user@avro.apache.org
Received: (qmail 72894 invoked by uid 99); 15 Oct 2013 23:50:30 -0000
Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136)
    by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 15 Oct 2013 23:50:30 +0000
X-ASF-Spam-Status: No, hits=1.5 required=5.0
	tests=HTML_MESSAGE,RCVD_IN_DNSWL_LOW,SPF_PASS
X-Spam-Check-By: apache.org
Received-SPF: pass (athena.apache.org: domain of ewasserman@247-inc.com
 designates 213.199.154.14 as permitted sender)
Received: from [213.199.154.14] (HELO
 emea01-am1-obe.outbound.protection.outlook.com) (213.199.154.14)
    by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 15 Oct 2013 23:50:25 +0000
Received: from SINPR03MB028.apcprd03.prod.outlook.com (10.242.50.153) by
 SINPR03MB026.apcprd03.prod.outlook.com (10.242.50.142) with Microsoft SMTP
 Server (TLS) id 15.0.785.10; Tue, 15 Oct 2013 23:50:00 +0000
Received: from SINPR03MB028.apcprd03.prod.outlook.com ([169.254.13.234]) by
 SINPR03MB028.apcprd03.prod.outlook.com ([169.254.13.90]) with mapi id
 15.00.0785.001; Tue, 15 Oct 2013 23:50:00 +0000
From: Eric Wasserman <ewasserman@247-inc.com>
To: "<user@avro.apache.org>" <user@avro.apache.org>
Subject: Re: GenericRecord and passivity
Thread-Topic: GenericRecord and passivity
Thread-Index: AQHOyfgYKxTBnagjj0agq40Zs/mbFZn2XM6AgAAEIgCAAA2iAA==
Date: Tue, 15 Oct 2013 23:49:59 +0000
Message-ID: <B4CFC519-4864-4C2E-B703-05C916ED5CCC@247-inc.com>
References: 
 <CAHnpetT40kkQ7G2nVfbp0p_PV9FpFRWNU22wjCPLZnD0spmM-w@mail.gmail.com>
 <CALEq1Z9L11FPYJE42MkO5XmFM0n4j+77JgUt6+etDDp-mCWy1Q@mail.gmail.com>
 <CAHnpetT_72u8NqSvPZ5SHXHPD9U1qPUeGtq04JbjKzzbumi2_Q@mail.gmail.com>
In-Reply-To: 
 <CAHnpetT_72u8NqSvPZ5SHXHPD9U1qPUeGtq04JbjKzzbumi2_Q@mail.gmail.com>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach: 
X-MS-TNEF-Correlator: 
x-originating-ip: [10.255.22.132]
x-forefront-prvs: 00003DBFE7
x-forefront-antispam-report: 
 SFV:NSPM;SFS:(199002)(189002)(377454003)(51704005)(164054003)(24454002)(47976001)(82746002)(16236675002)(50986001)(15395725003)(47736001)(74876001)(4396001)(51856001)(49866001)(15975445006)(15202345003)(54316002)(47446002)(19580395003)(63696002)(74662001)(59766001)(54356001)(56776001)(79102001)(77982001)(85306002)(74366001)(65816001)(31966008)(19580405001)(83322001)(80976001)(83072001)(56816003)(76796001)(76786001)(77096001)(81686001)(81816001)(33656001)(81342001)(69226001)(53806001)(81542001)(36756003)(46102001)(74706001)(10126625002)(491001);DIR:OUT;SFP:;SCL:1;SRVR:SINPR03MB026;H:SINPR03MB028.apcprd03.prod.outlook.com;CLIP:10.255.22.132;FPR:;RD:InfoNoRecords;A:1;MX:3;LANG:en;
Content-Type: multipart/alternative;
	boundary="_000_B4CFC51948644C2EB70305C916ED5CCC247inccom_"
MIME-Version: 1.0
X-OriginatorOrg: 247-inc.com
X-Virus-Checked: Checked by ClamAV on apache.org

--_000_B4CFC51948644C2EB70305C916ED5CCC247inccom_
Content-Type: text/plain; charset="iso-8859-1"
Content-Transfer-Encoding: quoted-printable

Change this line:
DatumReader<GenericRecord> reader =3D new GenericDatumReader<GenericRecord>=
(schema_11);

to this:
DatumReader<GenericRecord> reader =3D new GenericDatumReader<GenericRecord>=
(schema_10,  schema_11);


On Oct 15, 2013, at 4:01 PM, kulkarni.swarnim@gmail.com<mailto:kulkarni.swa=
rnim@gmail.com> wrote:

Old Schema: http://pastebin.com/x5CtmKWK
New Schema: http://pastebin.com/c8ZSKMaG
Code to deserialize: http://pastebin.com/RbBWm1VN


On Tue, Oct 15, 2013 at 5:46 PM, Doug Cutting <cutting@apache.org<mailto:cu=
tting@apache.org>> wrote:
GenericRecord should work well in this context.  Can you provide a
complete example that fails?

Doug

On Tue, Oct 15, 2013 at 3:43 PM, kulkarni.swarnim@gmail.com<mailto:kulkarni=
.swarnim@gmail.com>
<kulkarni.swarnim@gmail.com<mailto:kulkarni.swarnim@gmail.com>> wrote:
> Do we know if a GenericRecord is robust to schema evolution? I am current=
ly
> seeing cases where I get an exception like the following if I try to
> deserialize an older record with a newer schema.
>
> Exception in thread "main" java.lang.ArrayIndexOutOfBoundsException: 8
>
> at org.apache.avro.io.parsing.Symbol$Alternative.getSymbol(Symbol.java:36=
4)
>
> at org.apache.avro.io.ResolvingDecoder.doAction(ResolvingDecoder.java:229=
)
>
> at org.apache.avro.io.parsing.Parser.advance(Parser.java:88)
>
> at org.apache.avro.io.ResolvingDecoder.readIndex(ResolvingDecoder.java:20=
6)
>
> at
> org.apache.avro.generic.GenericDatumReader.read(GenericDatumReader.java:1=
52)
>
> at
> org.apache.avro.generic.GenericDatumReader.readRecord(GenericDatumReader.=
java:177)
>
> at
> org.apache.avro.generic.GenericDatumReader.read(GenericDatumReader.java:1=
48)
>
> at
> org.apache.avro.generic.GenericDatumReader.read(GenericDatumReader.java:1=
39)
>
>
> The newer schema just has a few more fields added to it and no names from
> the older schema were changed.
>
> Is this a known issue? Should a SpecificRecord be always considered when
> passivity is important?
>
>
> Thanks,
>
>
> --
> Swarnim



--
Swarnim


--_000_B4CFC51948644C2EB70305C916ED5CCC247inccom_
Content-Type: text/html; charset="iso-8859-1"
Content-ID: <D80690202DAB774E8DD35ACAFE1ACFF2@apcprd03.prod.outlook.com>
Content-Transfer-Encoding: quoted-printable

<html>
<head>
<meta http-equiv=3D"Content-Type" content=3D"text/html; charset=3Diso-8859-=
1">
</head>
<body style=3D"word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-lin=
e-break: after-white-space; ">
Change this line:
<div>
<div><span class=3D"Apple-tab-span" style=3D"white-space:pre"></span>DatumR=
eader&lt;GenericRecord&gt; reader =3D new GenericDatumReader&lt;GenericReco=
rd&gt;(schema_11);<br>
</div>
<div><br>
</div>
<div>
<div>
<div>to this:</div>
<div>
<div><span class=3D"Apple-tab-span" style=3D"white-space:pre"></span>DatumR=
eader&lt;GenericRecord&gt; reader =3D new GenericDatumReader&lt;GenericReco=
rd&gt;(schema_10, &nbsp;schema_11);<br>
</div>
<div><span class=3D"Apple-tab-span" style=3D"white-space:pre"></span></div>
<br>
</div>
<div><br>
</div>
<div>On Oct 15, 2013, at 4:01 PM, <a href=3D"mailto:kulkarni.swarnim@gmail.=
com">kulkarni.swarnim@gmail.com</a> wrote:</div>
<br class=3D"Apple-interchange-newline">
<blockquote type=3D"cite">
<div dir=3D"ltr">Old Schema:&nbsp;<a href=3D"http://pastebin.com/x5CtmKWK">=
http://pastebin.com/x5CtmKWK</a>
<div>New Schema:&nbsp;<a href=3D"http://pastebin.com/c8ZSKMaG">http://paste=
bin.com/c8ZSKMaG</a></div>
<div>Code to deserialize:&nbsp;<a href=3D"http://pastebin.com/RbBWm1VN">htt=
p://pastebin.com/RbBWm1VN</a></div>
</div>
<div class=3D"gmail_extra"><br>
<br>
<div class=3D"gmail_quote">On Tue, Oct 15, 2013 at 5:46 PM, Doug Cutting <s=
pan dir=3D"ltr">
&lt;<a href=3D"mailto:cutting@apache.org" target=3D"_blank">cutting@apache.=
org</a>&gt;</span> wrote:<br>
<blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1p=
x #ccc solid;padding-left:1ex">
GenericRecord should work well in this context. &nbsp;Can you provide a<br>
complete example that fails?<br>
<span class=3D"HOEnZb"><font color=3D"#888888"><br>
Doug<br>
</font></span>
<div class=3D"HOEnZb">
<div class=3D"h5"><br>
On Tue, Oct 15, 2013 at 3:43 PM, <a href=3D"mailto:kulkarni.swarnim@gmail.c=
om">kulkarni.swarnim@gmail.com</a><br>
&lt;<a href=3D"mailto:kulkarni.swarnim@gmail.com">kulkarni.swarnim@gmail.co=
m</a>&gt; wrote:<br>
&gt; Do we know if a GenericRecord is robust to schema evolution? I am curr=
ently<br>
&gt; seeing cases where I get an exception like the following if I try to<b=
r>
&gt; deserialize an older record with a newer schema.<br>
&gt;<br>
&gt; Exception in thread &quot;main&quot; java.lang.ArrayIndexOutOfBoundsEx=
ception: 8<br>
&gt;<br>
&gt; at org.apache.avro.io.parsing.Symbol$Alternative.getSymbol(Symbol.java=
:364)<br>
&gt;<br>
&gt; at org.apache.avro.io.ResolvingDecoder.doAction(ResolvingDecoder.java:=
229)<br>
&gt;<br>
&gt; at org.apache.avro.io.parsing.Parser.advance(Parser.java:88)<br>
&gt;<br>
&gt; at org.apache.avro.io.ResolvingDecoder.readIndex(ResolvingDecoder.java=
:206)<br>
&gt;<br>
&gt; at<br>
&gt; org.apache.avro.generic.GenericDatumReader.read(GenericDatumReader.jav=
a:152)<br>
&gt;<br>
&gt; at<br>
&gt; org.apache.avro.generic.GenericDatumReader.readRecord(GenericDatumRead=
er.java:177)<br>
&gt;<br>
&gt; at<br>
&gt; org.apache.avro.generic.GenericDatumReader.read(GenericDatumReader.jav=
a:148)<br>
&gt;<br>
&gt; at<br>
&gt; org.apache.avro.generic.GenericDatumReader.read(GenericDatumReader.jav=
a:139)<br>
&gt;<br>
&gt;<br>
&gt; The newer schema just has a few more fields added to it and no names f=
rom<br>
&gt; the older schema were changed.<br>
&gt;<br>
&gt; Is this a known issue? Should a SpecificRecord be always considered wh=
en<br>
&gt; passivity is important?<br>
&gt;<br>
&gt;<br>
&gt; Thanks,<br>
&gt;<br>
&gt;<br>
&gt; --<br>
&gt; Swarnim<br>
</div>
</div>
</blockquote>
</div>
<br>
<br clear=3D"all">
<div><br>
</div>
-- <br>
Swarnim </div>
</blockquote>
</div>
<br>
</div>
</div>
</body>
</html>

--_000_B4CFC51948644C2EB70305C916ED5CCC247inccom_--