crunch-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From David Ortiz <dor...@videologygroup.com>
Subject Strange ClassCastException emitting avro record
Date Mon, 23 Nov 2015 20:22:57 GMT
Hello,

     Anyone have any idea what might cause this stack trace?  I am trying to emit a descendent
of SpecificRecordBase that was generated using the maven avro plugin with an avdl file.  It
is all simple fields, no nested records or anything.  The last line in my code it references
is specifically the emitter.emit() call on my record instance.

Error: org.apache.crunch.CrunchRuntimeException: java.lang.ClassCastException: org.apache.avro.util.Utf8
cannot be cast to org.apache.avro.generic.IndexedRecord at org.apache.crunch.impl.mr.emit.MultipleOutputEmitter.emit(MultipleOutputEmitter.java:45)
at org.apache.crunch.MapFn.process(MapFn.java:34) at org.apache.crunch.impl.mr.run.RTNode.process(RTNode.java:98)
at org.apache.crunch.impl.mr.emit.IntermediateEmitter.emit(IntermediateEmitter.java:56) at
org.apache.crunch.MapFn.process(MapFn.java:34) at org.apache.crunch.impl.mr.run.RTNode.process(RTNode.java:98)
at org.apache.crunch.impl.mr.emit.IntermediateEmitter.emit(IntermediateEmitter.java:56) at
com.videologygroup.crunch.ClickstreamWarehouse$3.process(ClickstreamWarehouse.java:528) at
com.videologygroup.crunch.ClickstreamWarehouse$3.process(ClickstreamWarehouse.java:509) at
org.apache.crunch.impl.mr.run.RTNode.process(RTNode.java:98) at org.apache.crunch.impl.mr.emit.IntermediateEmitter.emit(IntermediateEmitter.java:56)
at org.apache.crunch.MapFn.process(MapFn.java:34) at org.apache.crunch.impl.mr.run.RTNode.process(RTNode.java:98)
at org.apache.crunch.impl.mr.emit.IntermediateEmitter.emit(IntermediateEmitter.java:56) at
org.apache.crunch.MapFn.process(MapFn.java:34) at org.apache.crunch.impl.mr.run.RTNode.process(RTNode.java:98)
at org.apache.crunch.impl.mr.run.RTNode.processIterable(RTNode.java:113) at org.apache.crunch.impl.mr.run.CrunchReducer.reduce(CrunchReducer.java:57)
at org.apache.hadoop.mapreduce.Reducer.run(Reducer.java:171) at org.apache.hadoop.mapred.ReduceTask.runNewReducer(ReduceTask.java:627)
at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:389) at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:163)
at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1671) at
org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158) Caused by: java.lang.ClassCastException:
org.apache.avro.util.Utf8 cannot be cast to org.apache.avro.generic.IndexedRecord at org.apache.avro.generic.GenericData.getField(GenericData.java:583)
at org.apache.avro.generic.GenericData.getField(GenericData.java:598) at org.apache.avro.generic.GenericDatumWriter.writeField(GenericDatumWriter.java:112)
at org.apache.avro.generic.GenericDatumWriter.writeRecord(GenericDatumWriter.java:104) at
org.apache.avro.generic.GenericDatumWriter.write(GenericDatumWriter.java:66) at org.apache.avro.generic.GenericDatumWriter.write(GenericDatumWriter.java:58)
at org.apache.crunch.types.avro.SafeAvroSerialization$AvroWrapperSerializer.serialize(SafeAvroSerialization.java:128)
at org.apache.crunch.types.avro.SafeAvroSerialization$AvroWrapperSerializer.serialize(SafeAvroSerialization.java:113)
at org.apache.hadoop.io.SequenceFile$BlockCompressWriter.append(SequenceFile.java:1517) at
org.apache.hadoop.mapreduce.lib.output.SequenceFileOutputFormat$1.write(SequenceFileOutputFormat.java:83)
at org.apache.crunch.io.CrunchOutputs$OutputState.write(CrunchOutputs.java:296) at org.apache.crunch.io.CrunchOutputs.write(CrunchOutputs.java:175)
at org.apache.crunch.impl.mr.emit.MultipleOutputEmitter.emit(MultipleOutputEmitter.java:41)
... 25 more

Thanks,
      Dave
This email is intended only for the use of the individual(s) to whom it is addressed. If you
have received this communication in error, please immediately notify the sender and delete
the original email.

Mime
View raw message