crunch-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jay Vyas <jayunit...@gmail.com>
Subject MemPipeline works, but running from files w/ MRPipeline fails.
Date Mon, 06 Jan 2014 03:48:27 GMT
Hi again crunch!

Well , although my MemPipeline implementations worked great.  But my
CrunchETL.java class is failing when i run it on a "real" file in my local
filesystem:  Here are the two ways that it breaks (link to the source is
also at the end of this email).


CrunchETL e = new CrunchETL(input,output).numberOfTransactionsByState()
results in this error:

1) Caused by: java.io.NotSerializableException:
org.bigtop.bigpetstore.etl.CrunchETL
    at java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1164)
    at
java.io.ObjectOutputStream.defaultWriteFields(ObjectOutputStream.java:1518)
    at
java.io.ObjectOutputStream.writeSerialData(ObjectOutputStream.java:1483)
    at
java.io.ObjectOutputStream.writeOrdinaryObject(ObjectOutputStream.java:1400)
    at java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1158)

To make this easy to debug, i added a main method into the CrunchETL.java
class, however that method fails with a different exception:

2) java.lang.NoSuchMethodError: org.apache.avro.mapred.AvroKey: method
<init>()V not found

** The link to the class is below **

https://github.com/jayunit100/bigpetstore/blob/d23ce6c7de2fcf94da76be4266a471c1b5de70ec/src/main/java/org/bigtop/bigpetstore/etl/CrunchETL.java

Any thoughts on why i get these failures?

Mime
View raw message