chukwa-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jiaqi Tan <tanji...@gmail.com>
Subject Running Hadoop MapReduce programs which link against Chukwa classes
Date Mon, 25 May 2009 12:38:11 GMT
Hi,

I'm trying to run a MapReduce program which links against Chukwa
classes e.g. ChukwaRecord, ChukwaRecordKey, ChukwaRecordPartitioner.
I've copied the chukwa-core-0.1.2.jar file into the lib/ directory of
the Hadoop installation that I'm running this program on. My code has
been successfully compiled (the build.xml for my program includes the
chukwa-core-0.1.2.jar file as one of the classpath entries), but when
I try to run the program, I'm getting error messages that look like:

2009-05-24 23:34:46,810 INFO org.apache.hadoop.metrics.jvm.JvmMetrics:
Initializing JVM Metrics with processName=MAP, sessionId=
2009-05-24 23:34:47,016 INFO org.apache.hadoop.mapred.MapTask: numReduceTasks: 1
2009-05-24 23:34:47,043 WARN org.apache.hadoop.mapred.TaskTracker:
Error running child
java.lang.RuntimeException: java.lang.RuntimeException:
java.lang.ClassNotFoundException:
org.apache.hadoop.chukwa.extraction.demux.ChukwaRecordPartitioner
        at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:680)
        at org.apache.hadoop.mapred.JobConf.getPartitionerClass(JobConf.java:739)
        at org.apache.hadoop.mapred.MapTask$MapOutputBuffer.<init>(MapTask.java:348)
        at org.apache.hadoop.mapred.MapTask.run(MapTask.java:193)
        at org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:2198)
Caused by: java.lang.RuntimeException:
java.lang.ClassNotFoundException:
org.apache.hadoop.chukwa.extraction.demux.ChukwaRecordPartitioner
        at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:648)
        at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:672)
        ... 4 more
Caused by: java.lang.ClassNotFoundException:
org.apache.hadoop.chukwa.extraction.demux.ChukwaRecordPartitioner
        at java.net.URLClassLoader$1.run(URLClassLoader.java:200)
        at java.security.AccessController.doPrivileged(Native Method)
        at java.net.URLClassLoader.findClass(URLClassLoader.java:188)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:307)
        at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:252)
        at java.lang.ClassLoader.loadClassInternal(ClassLoader.java:320)
        at java.lang.Class.forName0(Native Method)
        at java.lang.Class.forName(Class.java:247)
        at org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:628)
        at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:646)
        ... 5 more

I did have a line in my code that imports
org.apache.hadoop.chukwa.extraction.demux.* so I'm not sure if this is
a linking issue. Also, the mapper does actually start, and the input
does seem to be getting read, only that the partitioner is failing. Is
there something I'm missing?

Thanks,
Jiaqi

Mime
View raw message