crunch-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Micah Whitacre <mkwhita...@gmail.com>
Subject Re: serialize and materialize
Date Wed, 29 Apr 2015 22:52:59 GMT
Lucy,
  Can you provide a stack trace of the exceptions you are getting?  The
error might be with how Avro is trying to create a Schema for your class
using ReflectData[1]

[1] -
https://avro.apache.org/docs/1.7.6/api/java/org/apache/avro/reflect/ReflectData.html

On Wed, Apr 29, 2015 at 5:39 PM, Lucy Chen <lucychen2014fall@gmail.com>
wrote:

> Hi,
>
>      I have my own class defined as follows:
>
> public class A implements java.io.Serializable, Cloneable{
>
>  private String id;
>
> private String name;
>
> private Map<String, Float> codes;
>
>
> public A()
>
> {
>
> this(null, null, new HashMap<String, Float>());
>
> }
>
>
>   ............................
>
> }
>
>
> and will generate a PCollection<PTypeA> DataA, PType<A> PTypeA =
> Avros.records(A.class);
>
>
> Then when it is materialized by Iterable<PTypeA> DataA_mat.materialize(),
> it will throw a not serializable exception or not materializable exception.
> I think if I include Set<> as a member of class A instead of Map, it will
> fail too;
>
>
> Can any one knows how we can define the class appropriately if we want to
> materialize DataA here? Or is there another serialization way instead of
> Avros.records so that materialize() will work as well? I can only make it
> work for a class excluding a Set or Map type member. That makes a lot of
> trouble for me.
>
>
> Thanks.
>
>
> Lucy
>

Mime
View raw message