avro-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Ryan Blue (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (AVRO-1392) Unable to marshal enum to Avro
Date Wed, 06 Nov 2013 19:24:18 GMT

    [ https://issues.apache.org/jira/browse/AVRO-1392?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13815177#comment-13815177

Ryan Blue commented on AVRO-1392:

Is your code in a repository I can view somewhere? Or maybe you could post a gist? It would
be easier to follow if I could see where you serialize/deserialize, the reflected schema,
and any stack traces you encounter.

About the byte array to string problem: Are you sure you need to do this? Typically, you want
to use Avro to produce efficient binary files that are compressable and splittable (if that
makes no sense, just think of it as "doing the right thing" for you). That usually means that
you don't just use Avro to encode an object as a String, but you also use Avro to write an
entire file. But, it looks like you're trying to use Avro to encode a record, convert the
bytes to a String, and then store that in your own format. Could you go in to why you chose
to do it this way and what you want to accomplish? There is probably an easier way for you.

Now, I think I know why you don't see the enum. They are actually encoded, just not as text
strings. An enum is encoded in Avro as the index in the enum's schema. For example, look at
the enum I had before:
  "type" : "enum",
  "name" : "UserType",
  "symbols" : [ "ANNOYING", "ANGRY" ]

Because the symbols aren't going to change, Avro actually stores that as a number: 0 for "ANNOYING"
and 1 for "ANGRY" and so on if there were more symbols. So when you convert those bytes to
a String and write them to a file, you won't see the "ANNOYING" or "ANGRY", you'll see "\x00"
or "\x01".

> Unable to marshal enum to Avro
> ------------------------------
>                 Key: AVRO-1392
>                 URL: https://issues.apache.org/jira/browse/AVRO-1392
>             Project: Avro
>          Issue Type: Bug
>          Components: java
>         Environment: DEV
>            Reporter: Balaji Seshadri
>            Priority: Blocker
> I'm trying to marshall below object to Avro format and unable to marshall it using ReflectDatum
Writer.Enums are being rejected while marshalling.
> {code:java}
> public class AccountAccess extends DESBusinessEvent<String> {
>     @XmlElement(required = true)
>     protected String accountId;
>     @XmlElement(required = true)
>     @XmlSchemaType(name = "dateTime")
>     @AvroEncode(using=XmlDateAsLong.class)
>     protected XMLGregorianCalendar dateTime;
> //    @XmlElement(namespace = DESNamespaces.ENUM_NAMESPACE, required = true)
>     @XmlElement(required = true)
>     @Stringable
>     protected ApplicationEnum application;
> //    @XmlElement(namespace = DESNamespaces.ENUM_NAMESPACE, required = true)
>     @XmlElement(required = true)
>     @Stringable
>     protected UserTypeEnum userType;
>     protected String userId;
> }
> {code}

This message was sent by Atlassian JIRA

View raw message