avro-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From julianpeeters <julianpeet...@gmail.com>
Subject Re: Deserialize with different schema
Date Thu, 13 Aug 2015 08:49:53 GMT
Hi Mehrez,

Can I guess? You're reading some Python/Pig AvroStorage output? Hate that.

I get the same error when the reader schema has a namespace but the writer
has none. But only when a record is in a union.


Here's a pair of small runnable  examples
<https://github.com/julianpeeters/avro-namespace-issues/tree/master/reading>  
that show errors with reading and writing accross namespaces.

For the sake of being complete, here's my  question
<http://apache-avro.679487.n3.nabble.com/Issues-reading-and-writing-namespace-less-schemas-from-namespaced-Specific-Records-td4032092.html>

, and it looks like Vitaly Gordon ran into this issue as well,  here
<http://apache-avro.679487.n3.nabble.com/Unable-to-compile-a-namespace-less-schema-td4028318.html>

. 

IHMO this is a bug that hinders Avro's utility as a data interchange format.
I don't think the technical issue is in trying to import a class from the
default package (which succeeds outside of unions), but instead it's from
trying to resolve a union reflectively and the writer schema's fullname
doesn't match the class' fullname.

The fix for now:
You could try using the Generic API instead, and then map the Generic
Records to your Specific Records manually. Here's a start in Java:

        import org.apache.avro.Schema;
        import org.apache.avro.file.DataFileReader;
        import org.apache.avro.generic.GenericDatumReader;
        import org.apache.avro.generic.GenericRecord;

        GenericDatumReader<GenericRecord> datumReader = new
GenericDatumReader<>(schema);
        DataFileReader<GenericRecord> fileReader = new
DataFileReader<>(file, datumReader);
        GenericRecord record = fileReader.next();


Cheers,
Julian



--
View this message in context: http://apache-avro.679487.n3.nabble.com/Deserialize-with-different-schema-tp4032782p4032816.html
Sent from the Avro - Users mailing list archive at Nabble.com.

Mime
View raw message