mahout-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Mark <static.void....@gmail.com>
Subject Classpath question
Date Sat, 11 Sep 2010 01:44:26 GMT
  Perhaps this a better place to post this (Originally posted to Hadoop)


If I submit a jar that has a lib directory that contains a bunch of 
jars, shouldn't those jars be in the classpath and available to all nodes?

The reason I ask this is because I am trying to submit a jar myjar.jar 
that has the following structure

--src
  \.... (My source classes)
-- lib
   \
    -- mahout-collections-0.3.jar
    -- mahout-core-0.3.jar
    -- mahout-math-0.3.jar
    -- hbase-0.20.0.jar
    -- commons-cli-2.0-mahout.jar

Now the job I am trying to run is actually part of mahout-core-0.3... 
not src. The job starts but then fails with the following error

10/09/10 03:15:13 INFO mapred.JobClient: Task Id : 
attempt_201009100306_0003_r_000000_0, Status : FAILED
java.lang.RuntimeException: java.lang.reflect.InvocationTargetException
     at 
org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:115)
     at 
org.apache.hadoop.mapred.ReduceTask.runNewReducer(ReduceTask.java:550)
     at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:408)
     at org.apache.hadoop.mapred.Child.main(Child.java:170)
Caused by: java.lang.reflect.InvocationTargetException
     at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native 
Method)
     at 
sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
     at 
sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
     at java.lang.reflect.Constructor.newInstance(Constructor.java:532)
     at 
org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:113)
     ... 3 more
Caused by: java.lang.NoClassDefFoundError: 
org/apache/mahout/math/map/OpenObjectIntHashMap
     at 
org.apache.mahout.fpm.pfpgrowth.ParallelFPGrowthReducer.<init>(ParallelFPGrowthReducer.java:56)
     ... 8 more


Apparently it cant find the 
org/apache/mahout/math/map/OpenObjectIntHashMap.class although that 
class is definitely in mahout-collections-0.3.jar. Is this a problem 
because the mahout-core-0.3.jar doesn't have a lib directory?

What is an easy way around this?

Thanks





Mime
View raw message