avro-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Ryan Blue (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (AVRO-1554) Avro should have support for common constructs like UUID and Date
Date Thu, 26 Mar 2015 22:58:53 GMT

    [ https://issues.apache.org/jira/browse/AVRO-1554?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14382885#comment-14382885
] 

Ryan Blue commented on AVRO-1554:
---------------------------------

[~sachingoyal], I've been looking at the patch and thinking about ways to accomplish what
you're trying to do here. I'm not sure if reusing the CustomEncoding framework is a good idea,
although I think it's definitely a possibility because it doesn't introduce too many changes
overall. The problem is that I'd rather not make directly controlling the encoder the way
to implement high-level types.

It seems to me that something more like a combination between Stringable and CustomEncoding
is the right solution: Something that converts from a high-level type to a normal Avro type,
but with an API and registration. This would also fit with the "logical type" concept that
we introduced for decimals, dates, and times, where you have an underlying Avro type and optional
representations. What do you think?

> Avro should have support for common constructs like UUID and Date
> -----------------------------------------------------------------
>
>                 Key: AVRO-1554
>                 URL: https://issues.apache.org/jira/browse/AVRO-1554
>             Project: Avro
>          Issue Type: Bug
>          Components: java
>    Affects Versions: 1.7.6
>            Reporter: Sachin Goyal
>         Attachments: AVRO-1554.patch, AVRO-1554_2.patch, AVRO-1554_3.patch, CustomEncodingUnionBug.zip
>
>
> Consider the following code:
> {code}
> public class AvroExample
> {
>     public static void main (String [] args) throws Exception
>     {
>         ReflectData rdata = ReflectData.AllowNull.get();
>         Schema schema = rdata.getSchema(Temp.class);
>         
>         ReflectDatumWriter<Temp> datumWriter = 
>                new ReflectDatumWriter (Temp.class, rdata);
>         DataFileWriter<Temp> fileWriter = 
>                new DataFileWriter<Temp> (datumWriter);
>         ByteArrayOutputStream baos = new ByteArrayOutputStream();
>         fileWriter.create(schema, baos);
>         fileWriter.append(new Temp());
>         fileWriter.close();
>         byte[] bytes = baos.toByteArray();
>         GenericDatumReader<GenericRecord> datumReader = 
>                 new GenericDatumReader<GenericRecord> ();
>         SeekableByteArrayInput avroInputStream = 
>                 new SeekableByteArrayInput(bytes);
>         DataFileReader<GenericRecord> fileReader = 
>                 new DataFileReader<GenericRecord>(avroInputStream, datumReader);
>         schema = fileReader.getSchema();
>         GenericRecord record = null;
>         record = fileReader.next(record);
>         System.out.println (record);
>         System.out.println (record.get("id"));
>     }
> }
> class Temp
> {
>     UUID id = UUID.randomUUID();
>     Date date = new Date();
>     BigInteger bi = BigInteger.TEN;
> }
> {code}
> Output from this code is:
> {code:javascript}
> {"id": {}, "date": {}, "bi": "10"}
> {code}
> UUID and Date type fields are very common in Java and can be found a lot in third-party
code as well (where it may be difficult to put annotations).
> So Avro should include a default serialization/deserialization support for such fields.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message