crunch-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Som Satpathy <somsatpa...@gmail.com>
Subject Crunch on EMR
Date Tue, 01 Oct 2013 19:00:05 GMT
Hi All,

I have been trying to run crunch jobs on amazon EMR and faced a problem
while job execution -

"found class org.apache.hadoop.mapreduce.taskinputoutputcontext but
interface was expected"

This is happening because of hadoop incompatibilities between APIs used
while implementing the hadoop job, and the hadoop-code that runs in the
cluster.

My crunch fat jar is based on crunch version 0.7 (CDH 4.3.0) while EMR runs
hadoop 1.0.3 (where TaskInputOutputContext is implemented as an abstract
class)

Has any one been able to successfully execute their crunch jobs on EMR?

If yes, what are the best practices to make custom crunch fat jars work on
EMR?


Look forward to hearing your thoughts.

Thanks,

Som

Mime
View raw message