Mailing-List: contact dev-help@avro.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@avro.apache.org
Date: Tue, 20 Sep 2011 19:46:08 +0000 (UTC)
From: "Doug Cutting (JIRA)" <jira@apache.org>
To: dev@avro.apache.org
Message-ID: 
 <267758281.47432.1316547968858.JavaMail.tomcat@hel.zones.apache.org>
In-Reply-To: 
 <1171733126.45612.1316515388875.JavaMail.tomcat@hel.zones.apache.org>
Subject: [jira] [Commented] (AVRO-891) Exception when attempting to read
 modified schema containing deleted field
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit


    [ https://issues.apache.org/jira/browse/AVRO-891?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13108960#comment-13108960 ] 

Doug Cutting commented on AVRO-891:
-----------------------------------

Here are two ways we might avoid this:
  # Make the default constructor for SpecificDatumReader protected instead of public.  That would force folks to always provide a schema and would be incompatible.
  # Change SpecificDatumReader#setSchema() to, when the expected schema is null, set it to getSpecificData().getClass(actual).getSchema().  That would get the schema with the same name as the one in the file from the currently loaded class.

I prefer the second option. 

> Exception when attempting to read modified schema containing deleted field
> --------------------------------------------------------------------------
>
>                 Key: AVRO-891
>                 URL: https://issues.apache.org/jira/browse/AVRO-891
>             Project: Avro
>          Issue Type: Bug
>          Components: java
>    Affects Versions: 1.5.4
>         Environment: OSX 10.7
>            Reporter: Alex Holmes
>
> An AvroRuntimeException exception is thrown when attempting to read an Avro file serialized with an older version of a schema containing a field which has been subsequently removed in the newer schema.
> *Exception*
> {code}
> Exception in thread "main" org.apache.avro.AvroRuntimeException: Bad index
> 	at Record.put(Unknown Source)
> 	at org.apache.avro.generic.GenericData.setField(GenericData.java:463)
> 	at org.apache.avro.generic.GenericDatumReader.readRecord(GenericDatumReader.java:166)
> 	at org.apache.avro.generic.GenericDatumReader.read(GenericDatumReader.java:138)
> 	at org.apache.avro.generic.GenericDatumReader.read(GenericDatumReader.java:129)
> 	at org.apache.avro.file.DataFileStream.next(DataFileStream.java:233)
> 	at org.apache.avro.file.DataFileStream.next(DataFileStream.java:220)
> 	at Read.readFromAvro(Unknown Source)
> 	at Read.main(Unknown Source)
> {code}
> *Steps to reproduce*
> #  Generate code for schema v1 and v2
> #  Write an Avro file with the v1 code-generated Record class using the DataFileWriter and SpecificDatumWriter
> #  (informational only) Read the Avro file using the v1 code-generated Record class using DataFileStream and SpecificDatumReader (output follows)
> {code}
> Record@2ec791b9[name=r1,id=1]
> Record@bd86fd3[name=r2,id=2]
> {code}
> #  Read the Avro file using the v2 code-generated Record class using DataFileStream and SpecificDatumReader
> *Schema details*
> v1 schema:
> {code}
> {"name": "Record", "type": "record",
>   "fields": [
>     {"name": "name", "type": "string"},
>     {"name": "id", "type": "int"}
>   ]
> }
> {code}
> v2 schema:
> {code}
> {"name": "Record", "type": "record",
>   "fields": [
>     {"name": "name", "type": "string"}
>   ]
> }
> {code}
> *Write code*
> {code}
>   public static Record createRecord(String name, int id) {
>     Record record = new Record();
>     record.name = name;
>     record.id = id;
>     return record;
>   }
>   public static void writeToAvro(OutputStream outputStream)
>       throws IOException {
>     DataFileWriter<Record> writer =
>         new DataFileWriter<Record>(new SpecificDatumWriter<Record>());
>     writer.create(Record.SCHEMA$, outputStream);
>     writer.append(createRecord("r1", 1));
>     writer.append(createRecord("r2", 2));
>     writer.close();
>     outputStream.close();
>   }
> {code}
> *Read code*
> {code}
>   public static void readFromAvro(InputStream is) throws IOException {
>     DataFileStream<Record> reader = new DataFileStream<Record>(
>             is, new SpecificDatumReader<Record>());
>     for (Record a : reader) {
>       System.out.println(ToStringBuilder.reflectionToString(a));
>     }
>     IOUtils.cleanup(null, is);
>     IOUtils.cleanup(null, reader);
>   }
> {code}

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira