crunch-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jay Vyas <jayunit...@gmail.com>
Subject Two strange crunch exceptions
Date Sat, 15 Feb 2014 02:29:03 GMT
Hi crunch !

When running MRPipeline here:
https://github.com/jayunit100/bigpetstore/blob/63958ddde9eb18ea75f078668a0023288547bbcf/src/integration/java/org/bigtop/bigpetstore/integration/BigPetStoreCrunchIT.java

I got two errors:

1) Running the code as-is, I got this very strange stacktrace:

Exception in thread "Thread-3" java.lang.VerifyError: class
org.apache.hadoop.yarn.proto.YarnProtos$URLProto overrides final method
getUnknownFields.()Lcom/google/protobuf/UnknownFieldSet;
...

So, to debug... I tried something else and got another strange error.....

2)  I changed line 45  to a "MemPipeline", and a new error took its place:

java.lang.IncompatibleClassChangeError: Found interface
org.apache.hadoop.mapreduce.TaskInputOutputContext, but class was expected
    at org.apache.crunch.DoFn.getConfiguration(DoFn.java:127)
    at
org.apache.crunch.fn.Aggregators$AggregatorCombineFn.initialize(Aggregators.java:471)
    at ...

I assume there is something wrong with my environment or job setup, but ive
logically used most of this code before, and am just refactoring, so am
puzzled why all these funny errors have cropped up.

any thoughts or leads would be very much appreciated ....

-- 
Jay Vyas
http://jayunit100.blogspot.com

Mime
View raw message