crunch-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From David Ortiz <dpo5...@gmail.com>
Subject Re: Strange ClassCastException emitting avro record
Date Mon, 23 Nov 2015 21:38:11 GMT
Disregard.  I forgot to specify Writables.strings() as my output type when
writing to sequence file.

On Mon, Nov 23, 2015 at 3:23 PM David Ortiz <dortiz@videologygroup.com>
wrote:

> Hello,
>
>
>
>      Anyone have any idea what might cause this stack trace?  I am trying
> to emit a descendent of SpecificRecordBase that was generated using the
> maven avro plugin with an avdl file.  It is all simple fields, no nested
> records or anything.  The last line in my code it references is
> specifically the emitter.emit() call on my record instance.
>
>
>
> Error: org.apache.crunch.CrunchRuntimeException:
> java.lang.ClassCastException: org.apache.avro.util.Utf8 cannot be cast to
> org.apache.avro.generic.IndexedRecord at
> org.apache.crunch.impl.mr.emit.MultipleOutputEmitter.emit(MultipleOutputEmitter.java:45)
> at org.apache.crunch.MapFn.process(MapFn.java:34) at
> org.apache.crunch.impl.mr.run.RTNode.process(RTNode.java:98) at
> org.apache.crunch.impl.mr.emit.IntermediateEmitter.emit(IntermediateEmitter.java:56)
> at org.apache.crunch.MapFn.process(MapFn.java:34) at
> org.apache.crunch.impl.mr.run.RTNode.process(RTNode.java:98) at
> org.apache.crunch.impl.mr.emit.IntermediateEmitter.emit(IntermediateEmitter.java:56)
> at
> com.videologygroup.crunch.ClickstreamWarehouse$3.process(ClickstreamWarehouse.java:528)
> at
> com.videologygroup.crunch.ClickstreamWarehouse$3.process(ClickstreamWarehouse.java:509)
> at org.apache.crunch.impl.mr.run.RTNode.process(RTNode.java:98) at
> org.apache.crunch.impl.mr.emit.IntermediateEmitter.emit(IntermediateEmitter.java:56)
> at org.apache.crunch.MapFn.process(MapFn.java:34) at
> org.apache.crunch.impl.mr.run.RTNode.process(RTNode.java:98) at
> org.apache.crunch.impl.mr.emit.IntermediateEmitter.emit(IntermediateEmitter.java:56)
> at org.apache.crunch.MapFn.process(MapFn.java:34) at
> org.apache.crunch.impl.mr.run.RTNode.process(RTNode.java:98) at
> org.apache.crunch.impl.mr.run.RTNode.processIterable(RTNode.java:113) at
> org.apache.crunch.impl.mr.run.CrunchReducer.reduce(CrunchReducer.java:57)
> at org.apache.hadoop.mapreduce.Reducer.run(Reducer.java:171) at
> org.apache.hadoop.mapred.ReduceTask.runNewReducer(ReduceTask.java:627) at
> org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:389) at
> org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:163) at
> java.security.AccessController.doPrivileged(Native Method) at
> javax.security.auth.Subject.doAs(Subject.java:415) at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1671)
> at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158) Caused by:
> java.lang.ClassCastException: org.apache.avro.util.Utf8 cannot be cast to
> org.apache.avro.generic.IndexedRecord at
> org.apache.avro.generic.GenericData.getField(GenericData.java:583) at
> org.apache.avro.generic.GenericData.getField(GenericData.java:598) at
> org.apache.avro.generic.GenericDatumWriter.writeField(GenericDatumWriter.java:112)
> at
> org.apache.avro.generic.GenericDatumWriter.writeRecord(GenericDatumWriter.java:104)
> at
> org.apache.avro.generic.GenericDatumWriter.write(GenericDatumWriter.java:66)
> at
> org.apache.avro.generic.GenericDatumWriter.write(GenericDatumWriter.java:58)
> at
> org.apache.crunch.types.avro.SafeAvroSerialization$AvroWrapperSerializer.serialize(SafeAvroSerialization.java:128)
> at
> org.apache.crunch.types.avro.SafeAvroSerialization$AvroWrapperSerializer.serialize(SafeAvroSerialization.java:113)
> at
> org.apache.hadoop.io.SequenceFile$BlockCompressWriter.append(SequenceFile.java:1517)
> at
> org.apache.hadoop.mapreduce.lib.output.SequenceFileOutputFormat$1.write(SequenceFileOutputFormat.java:83)
> at
> org.apache.crunch.io.CrunchOutputs$OutputState.write(CrunchOutputs.java:296)
> at org.apache.crunch.io.CrunchOutputs.write(CrunchOutputs.java:175) at
> org.apache.crunch.impl.mr.emit.MultipleOutputEmitter.emit(MultipleOutputEmitter.java:41)
> ... 25 more
>
>
>
> Thanks,
>
>       Dave
> *This email is intended only for the use of the individual(s) to whom it
> is addressed. If you have received this communication in error, please
> immediately notify the sender and delete the original email.*
>

Mime
View raw message