crunch-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Hrishikesh P <hrishi.engin...@gmail.com>
Subject ClassCastException: org.apache.avro.util.Utf8 cannot be cast to org.apache.avro.generic.IndexedRecord
Date Thu, 06 Aug 2015 15:46:11 GMT
I'm trying to write a PTable<String, Map<String, String>> to a
TextFileSourceTarget but I am running into a ClassCastException.

{code}
PTable<String, Map<String, String>> table = ...;

table.write(new
TextFileSourceTarget<>(Path.mergePaths(fs.getHomeDirectory(), path),
Avros.tableOf(Avros.strings(), Avros.maps(Avros.strings()))));
{code}

The exception is:

WARN [main] org.apache.hadoop.mapred.YarnChild: Exception running
child : java.lang.ClassCastException: org.apache.avro.util.Utf8 cannot
be cast to org.apache.avro.generic.IndexedRecord
	at org.apache.crunch.types.avro.AvroTableType$IndexedRecordToPair.map(AvroTableType.java:88)
	at org.apache.crunch.MapFn.process(MapFn.java:34)
	at org.apache.crunch.impl.mr.run.RTNode.process(RTNode.java:98)
	at org.apache.crunch.impl.mr.run.RTNode.process(RTNode.java:109)
	at org.apache.crunch.impl.mr.run.CrunchMapper.map(CrunchMapper.java:60)
	at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:145)
	at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:787)
	at org.apache.hadoop.mapred.MapTask.run(MapTask.java:341)
	at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:163)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.security.auth.Subject.doAs(Subject.java:422)
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1671)
	at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158)


Does someone have an idea why the exception might be occurring? The version
of avro is 1.7.6. I have checked for dependency conflicts but there doesn't
seem to be a problem with it although it looks like it.

Thanks.

Mime
View raw message