crunch-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Kaspar Fischer <kaspar.fisc...@dreizak.com>
Subject POJO serialization using generics
Date Wed, 05 Nov 2014 06:50:12 GMT
Hi everybody,

I am using Crunch with Avros serialization. Some intermediate objects of my
pipeline are POJO's that have fields with generics. When running the
pipeline I get:

Exception in thread "main" org.apache.avro.AvroTypeException: Unknown type:
T

at org.apache.avro.specific.SpecificData.createSchema(SpecificData.java:220)

at org.apache.avro.reflect.ReflectData.createSchema(ReflectData.java:374)

at org.apache.avro.reflect.ReflectData.createSchema(ReflectData.java:283)

at
org.apache.avro.reflect.ReflectData.createFieldSchema(ReflectData.java:430)

at
org.apache.avro.reflect.ReflectData$AllowNull.createFieldSchema(ReflectData.java:72)

at org.apache.avro.reflect.ReflectData.createSchema(ReflectData.java:354)

at org.apache.avro.specific.SpecificData.createSchema(SpecificData.java:197)

at org.apache.avro.reflect.ReflectData.createSchema(ReflectData.java:374)

at org.apache.avro.reflect.ReflectData.createSchema(ReflectData.java:283)

at
org.apache.avro.reflect.ReflectData.createFieldSchema(ReflectData.java:430)

at
org.apache.avro.reflect.ReflectData$AllowNull.createFieldSchema(ReflectData.java:72)

at org.apache.avro.reflect.ReflectData.createSchema(ReflectData.java:354)

at org.apache.avro.specific.SpecificData.getSchema(SpecificData.java:154)

at org.apache.crunch.types.avro.Avros.reflects(Avros.java:275)

at org.apache.crunch.types.avro.Avros.containers(Avros.java:264)

at org.apache.crunch.types.avro.Avros.records(Avros.java:252)

How can I get around this? Is there a way to serialize the object using
Java serialization (or Kryo)?

Thanks for any pointers,
Kaspar

Mime
View raw message