avro-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Burak Emre <emrekaba...@gmail.com>
Subject Adding new field with default value to an Avro schema
Date Tue, 03 Feb 2015 11:57:47 GMT
> I added a field with a default value to an Avro schema which is previously used for writing
data. Is it possible to read the previous data using only new schema which has that new field
at the end?
> 
> I tried this scenario but unfortunately it throws EOFException while reading third field.
Even though it has a default value and the previous fields is read successfully, I'm not able
to de-serialize the record back without providing the writer schema I used previously.
> 
> Schema schema = Schema.createRecord("test", null, "avro.test", false); schema.setFields(Lists.newArrayList(
new Field("project", Schema.create(Type.STRING), null, null), new Field("city", Schema.createUnion(Lists.newArrayList(Schema.create(Type.NULL),
Schema.create(Type.STRING))), null, NullNode.getInstance()) )); GenericData.Record record
= new GenericRecordBuilder(schema) .set("project", "ff").build(); GenericDatumWriter w = new
GenericDatumWriter(schema); ByteArrayOutputStream outputStream = new ByteArrayOutputStream();
BinaryEncoder encoder = EncoderFactory.get().binaryEncoder(outputStream, null); w.write(record,
encoder); encoder.flush(); schema = Schema.createRecord("test", null, "avro.test", false);
schema.setFields(Lists.newArrayList( new Field("project", Schema.create(Type.STRING), null,
null), new Field("city", Schema.createUnion(Lists.newArrayList(Schema.create(Type.NULL), Schema.create(Type.STRING))),
null, NullNode.getInstance()), new Field("newField", Schema.createUni
on(Lists.newArrayList(Schema.create(Type.NULL), Schema.create(Type.STRING))), null, NullNode.getInstance())
)); DatumReader<GenericRecord> reader = new GenericDatumReader<>(schema); Decoder
decoder = DecoderFactory.get().binaryDecoder(outputStream.toByteArray(), null); GenericRecord
result = reader.read(null, decoder);
> 



Mime
View raw message