mahout-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Matt Spitz <msp...@meebo-inc.com>
Subject Classpath when running over Hadoop
Date Mon, 08 Nov 2010 21:11:24 GMT
Hey, folks,

I've written my own custom sequence file loader, and I'm wondering how to
use it in a hadoop job.

According to this:
http://www.mail-archive.com/hadoop-user@lucene.apache.org/msg01762.html

I should just be able to put it in the mahout-0.4/lib directory, right?  I
did this and modified bin/mahout to print the classpath it runs mahout with
 to confirm that the JAR I added was on the classpath (it is).

I'm still getting a CNFE exception when trying to run it as a hadoop job.
 Running locally is fine, as setting CLASSPATH=urlclustering.jar when
running bin/mahout works like a charm.

10/11/08 21:09:39 WARN driver.MahoutDriver: Unable to add class:
com.mspitz.urlclustering.BagsToSequenceFile
java.lang.ClassNotFoundException:
com.mspitz.urlclustering.BagsToSequenceFile
        at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
        at java.security.AccessController.doPrivileged(Native Method)
        at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:307)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:248)
        at java.lang.Class.forName0(Native Method)
        at java.lang.Class.forName(Class.java:169)
        at
org.apache.mahout.driver.MahoutDriver.addClass(MahoutDriver.java:207)
        at org.apache.mahout.driver.MahoutDriver.main(MahoutDriver.java:124)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
        at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
        at java.lang.reflect.Method.invoke(Method.java:597)
        at org.apache.hadoop.util.RunJar.main(RunJar.java:186)

Any thoughts?

Thanks,
Matt

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message