avro-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Poole, Samuel [USA]" <poole_sam...@bah.com>
Subject RE: Java Example of writing a union
Date Mon, 08 Aug 2011 20:07:13 GMT
Figured out the read problem.  Of course I had an if statement where I need a while. :)

From: Poole, Samuel [USA] [poole_samuel@bah.com]
Sent: Monday, August 08, 2011 3:56 PM
To: user@avro.apache.org
Subject: RE: Java Example of writing a union

Thank you very much. Yes, this works good. And then I took it one step further to try and
get the schema put in the file and also to apply encoding.

FOO fooObj = ....
BAR barObj = ....
BAR barObj2 = ....
        ByteArrayOutputStream out = new ByteArrayOutputStream();
        DatumWriter<SpecificRecord> writer = new SpecificDatumWriter<SpecificRecord>(yourSchema);

        DataFileWriter filewriter=new DataFileWriter(writer);

        CodecFactory codec = CodecFactory.deflateCodec(9);



        encoder = EncoderFactory.get().binaryEncoder(out, encoder);




        OutputStream outstream=new FileOutputStream("/somefolder/somefile.avro");


this code works, but now I have an issue with reading the file....

When I read the file, I can only see the first datum in the union.  I know that all of the
datums were written to the file because of the size of the file, but I can't read all of the

Here is my code to read the union file.

Schema yourSchema=Schema.parse(new File("/somefolder/someschema.avro"));

DatumReader<SpecificRecord> datumreader=new SpecificDatumReader<SpecificRecord>(yourSchema);

DataFileReader reader=new DataFileReader(new File("/somefolder/somefile.avro"),datumreader);

while (reader.hasNext()){

    SpecificRecord result=(SpecificRecord) reader.next();



Not sure if I have a problem with how I created the file or how I am reading the file....

Any ideas?


From: Vyacheslav Zholudev [vyacheslav.zholudev@gmail.com]
Sent: Monday, August 08, 2011 12:52 PM
To: user@avro.apache.org
Subject: Re: Java Example of writing a union

I'm assuming for now that you are using a specific writer and you have a union schema with
two records FOO and BAR (you should get two classes FOO and BAR generated by avro tools):

FOO fooObj = ....
BAR barObj = ....
BAR barObj2 = ....
        ByteArrayOutputStream out = new ByteArrayOutputStream();
        DatumWriter<GenericRecord> writer = new SpecificDatumWriter<Record>(yourSchema);
        encoder = EncoderFactory.get().binaryEncoder(out, encoder);
        writer.write(fooObj, encoder);
        writer.write(barObj, encoder);
        writer.write(barObj2, encoder);

Does it make sense?


On Aug 8, 2011, at 3:53 PM, Sam Poole wrote:

Does anybody have an example of writing a file that uses a union schema?  I
am having problems trying to write a file that uses a union schema because
once I set the schema, I can't add an individual datum because it is not
part of a union.

View this message in context: http://apache-avro.679487.n3.nabble.com/Java-Example-of-writing-a-union-tp3235624p3235624.html
Sent from the Avro - Users mailing list archive at Nabble.com<http://Nabble.com>.

View raw message