avro-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jacob Maes <jm...@linkedin.com>
Subject ArrayIndexOutOfBoundsException while writing an old sub record to an evolved superrecord
Date Wed, 25 Mar 2015 01:31:52 GMT
Hello,

I’m currently working on a system that receives Avro messages via Kafka which contain message
records and then writes them to a persistent store using a schema that is a super record (contains
all the schemas of the message records).

Recently, I have had to evolve the persistence schema to include a couple extra (optional)
fields. After making this schema change, I get an ArrayIndexOutOfBoundsException when the
system receives an old version of the message and tries to inject it into the new version
of the schema.

I’m attaching a test case that demonstrates the issue. The testAvroNormalUpdate() case passes
while the testAvroEvolvedUpdate() case produces the stack trace below.

The new schema should be backward compatible, so I’m not sure where I went wrong.  I’m
using Avro 1.4, but I’ve confirmed the same issue on Avro 1.7.7.

Any help would be much appreciated.

Thanks,
Jake

java.lang.ArrayIndexOutOfBoundsException: 2
at org.apache.avro.generic.GenericData$Record.get(GenericData.java:64)
at org.apache.avro.generic.GenericDatumWriter.getField(GenericDatumWriter.java:113)
at org.apache.avro.generic.GenericDatumWriter.writeRecord(GenericDatumWriter.java:100)
at org.apache.avro.generic.GenericDatumWriter.write(GenericDatumWriter.java:64)
at org.apache.avro.generic.GenericDatumWriter.writeArray(GenericDatumWriter.java:133)
at org.apache.avro.generic.GenericDatumWriter.write(GenericDatumWriter.java:66)
at org.apache.avro.generic.GenericDatumWriter.write(GenericDatumWriter.java:71)
at org.apache.avro.generic.GenericDatumWriter.writeRecord(GenericDatumWriter.java:102)
at org.apache.avro.generic.GenericDatumWriter.write(GenericDatumWriter.java:64)
at org.apache.avro.generic.GenericDatumWriter.write(GenericDatumWriter.java:56)
at com.pub.avro.TestEvolvedSchema.genericRecordToJson(TestEvolvedSchema.java:234)
at com.pub.avro.TestEvolvedSchema.genericRecordToJson(TestEvolvedSchema.java:240)
at com.pub.avro.TestEvolvedSchema.testAvroEvolvedUpdate(TestEvolvedSchema.java:190)

Mime
View raw message