avro-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Douglas Campbell <deegs...@yahoo.com>
Subject Re: Best way to read and write bytes of Avro object
Date Mon, 30 Nov 2009 19:42:18 GMT
Any thoughts on this?



----- Original Message ----
From: Douglas Campbell <deegs_ca@yahoo.com>
To: avro-user@hadoop.apache.org
Sent: Wed, November 25, 2009 7:31:02 PM
Subject: Best way to read and write bytes of Avro object

I'm using java 1.6 and avro 1.2.

My use case is to be able to create an avro object, serialize/marshall it into a byte array
and base64 it, and then un-base64 it, desearialize/unmarshall it back into the AvroObject.

Pretty simple right?  This the code I needed to write to do that .  Is there a better way
or utility classes in Avro java api which do this (leaving out the base64 stuff perhaps)?
   

/** decodes base64'd data into bytes and then re-creates an avro object */
public AvroModel fromAvroData(String base64) throws IOException
    {
        AvroModel am     = new AvroModel();
        final byte[] avrobytes = new Base64().decode(base64.getBytes());
        DatumReader dr   = new SpecificDatumReader(am.getSchema());
        SeekableByteArrayInputStream bin = new SeekableByteArrayInputStream(avrobytes);
        DataFileReader<AvroModel> dfr = new DataFileReader<AvroModel>(bin, dr);
        dfr.next(am);
        return am;
    }

    /**
        Converts an AvroModel to avro base64 encoded string 
    */
    public String toAvroData(AvroModel m) throws IOException
    {  
        DatumWriter dw = new SpecificDatumWriter(m.getSchema());
        ByteArrayOutputStream bout = new ByteArrayOutputStream();
        DataFileWriter<AvroModel> dfw = new DataFileWriter<AvroModel>(m.getSchema(),
bout, dw);
        dfw.append(m);
        dfw.flush();

        return new String(new Base64().encode(bout.toByteArray()));
    }

Note that for the "fromAvroData", I had to provide an implementation of SeekableInput over
a ByteArrayInputStream.

Thanks for the pointers.


      

Mime
View raw message