hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Steve Loughran <ste...@apache.org>
Subject Re: Maven and Mini MR Cluster
Date Thu, 04 Feb 2010 11:55:19 GMT
Michael Basnight wrote:
> Im using maven to run all my unit tests, and i have a unit test that creates a mini mr
cluster. When i create this cluster, i get classdefnotfound errors for the core hadoop libs
(Caused by: java.lang.ClassNotFoundException: org.apache.hadoop.mapred.Child). When i run
the same test w/o creating the mini cluster, well.. it works fine. My HADOOP_HOME is set to
the same version as my mvn repo, and points to a valid installation of hadoop. When i validate
the classpath thru maven, (dependency:build-classpath), it says that the core libs are on
the classpath as well (sourced from my .m2 repository). I just cant figure out why hadoop's
mini cluster cant find those jars. Running hadoop 0.20.0. 
> 
> Any suggestions?

the miniMR cluster does everything in memory, and doesnt look at 
HADOOP_HOME, which is only for the shell scripts.

It sounds like you need hadoop-mapreduce on your classpath. Sounds like. 
the Child class is the entry point used when creating new JVMs, and it 
is that classpath that isn't right, which is a forked JVM from the one 
the MiniMRCluster was created in.

Mime
View raw message