avro-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Sean Busbey <bus...@cloudera.com>
Subject Re: serialization-deserialization problem
Date Thu, 04 Jun 2015 04:30:15 GMT
The JSON listed is not the form that Avro's json encoder/decoder can
handle. Because the optional fields are unions, Avro's decoder expects you
to first list the type before the values.

Presuming CustomerEmails is an Avro record.
i.e.

{"emails" : { "CustomerEmails" : { "emails": ["a@a.com", "b@b.com"]} },
"transactions": null, "features" : null }

See the spec for more details:
http://avro.apache.org/docs/1.7.7/spec.html#json_encoding

To use the JsonDecoder, you will have to use a JsonEncoder rather than
relying on the toString method (the format of its output is undefined, so
that's probably a good idea anyways).

On Wed, Jun 3, 2015 at 5:20 PM, C 4.5 <cfourfive@gmail.com> wrote:

> Hi All,
>
> I have defined a simple Avro schema for Customer objects. Schema compiles
> OK and don't have any problem.
>
> I am trying to serialize to json string and deserialize into Customer java
> objects but I am not able to.
>
> Please see below:
> [1]: is the full code snippet
> [2]: is the output I obtain
>
> I am using Avro 1.7.7
>
> Customer's schema is flows. Emails are optional.
> {"namespace": "...",
>  "type": "record",
>  "name": "Customer",
>  "fields": [
>      {"name": "emails",       "type": [ "null", "....CustomerEmails"]},
>      {"name": "transactions", "type": [ "null",
> "...CustomerTransactions"]},
>      {"name": "features", "type": [ "null", "...CustomerFeatures"]}
>  ]
> }
>
> I have researched and tried different things but something seems to escape
> me, this should be a very trivial case to support.
> Hence, I am sure I am just overlooking something that is probably obvious.
>
> Any feedback?
> Thanks a ton
>
> ===[1] code
>            // ser
>            List<CharSequence> emails = new ArrayList<CharSequence>();
>             emails.add("a@a.com");
>             emails.add("b@b.com");
>
>             CustomerEmails customerEmails = CustomerEmails.newBuilder().
>                     setEmails(emails).
>                     build();
>
>             Customer customer = Customer.newBuilder().
>                     setEmails(customerEmails).
>                     setTransactions(null).
>                     setFeatures(null).
>                     build();
>
>             String customerJson = customer.toString();
>             System.out.println("customer json: " + customerJson);
>
>             // de-ser
>             JsonDecoder jsonDecoder =
> DecoderFactory.get().jsonDecoder(Customer.getClassSchema(), customerJson);
>             SpecificDatumReader<Customer> reader = new
> SpecificDatumReader<Customer>(Customer.class);
>             Object obj = reader.read(null, jsonDecoder);
>
>             System.out.println("obj=" + obj);
>
> ===[2] output
> customer json: {"emails": {"emails": ["a@a.com", "b@b.com"]},
> "transactions": null, "features": null}
> Exception in thread "main" org.apache.avro.AvroTypeException: Unknown
> union branch emails
>     at org.apache.avro.io.JsonDecoder.readIndex(JsonDecoder.java:445)
>     at
> org.apache.avro.io.ResolvingDecoder.doAction(ResolvingDecoder.java:290)
>     at org.apache.avro.io.parsing.Parser.advance(Parser.java:88)
>     at
> org.apache.avro.io.ResolvingDecoder.readIndex(ResolvingDecoder.java:267)
>     at
> org.apache.avro.generic.GenericDatumReader.read(GenericDatumReader.java:155)
>     at
> org.apache.avro.generic.GenericDatumReader.readField(GenericDatumReader.java:193)
>     at
> org.apache.avro.generic.GenericDatumReader.readRecord(GenericDatumReader.java:183)
>     at
> org.apache.avro.generic.GenericDatumReader.read(GenericDatumReader.java:151)
>     at
> org.apache.avro.generic.GenericDatumReader.read(GenericDatumReader.java:142)
>     at com.mytest.Deser.main(Deser.java:79)
>     at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>     at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
>     at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>     at java.lang.reflect.Method.invoke(Method.java:497)
>



-- 
Sean

Mime
View raw message