hadoop-mapreduce-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Wilkes, Chris" <cwil...@gmail.com>
Subject How can I compile mapreduce to use my own version of hadoop-core 0.21-SNAPSHOT?
Date Mon, 04 Jan 2010 20:05:18 GMT
I'm trying to build the 0.21.0-SNAPSHOT packages of common, hdfs, and  
mapreduce and am doing so with "ant binary" (I'm more used to maven so  
I'm not very good at debugging this), I've also done a mvn-install to  
put the jars into the local maven repo.

"ant binary" in seems to pull down the hdfs-core.jar file from the  
Hudson server:
   java -cp ./build/hadoop-mapred-0.21.0-SNAPSHOT/lib/hadoop- 
core-0.21.0-SNAPSHOT.jar org.apache.hadoop.util.VersionInfo
   Hadoop 0.21.0-SNAPSHOT
   Subversion http://svn.apache.org/repos/asf/hadoop/common/branches/branch-0.21 
  -r 893667
   Compiled by hudson on Thu Dec 24 09:34:21 UTC 2009
   From source with checksum 337d7f1a109e0ac3c7d2874300ce9b78
and not my one.  Also it pulls down the 0.22-SNAPSHOT one as well.

Perhaps I shouldn't be concerned about this but I'm always getting  
this error on the tasktracker startup:

2010-01-04 11:55:33,090 ERROR org.apache.hadoop.mapred.TaskTracker:  
Can not start task tracker because java.lang.NoSuchMethodError:  
org.apache.hadoop.ipc.RPC.waitForProxy(Ljava/lang/Class;JLjava/net/ 
InetSocketAddress;Lorg/apache/hadoop/conf/Configuration;)Ljava/lang/ 
Object;
         at  
org.apache.hadoop.mapred.TaskTracker.initialize(TaskTracker.java:601)
         at  
org.apache.hadoop.mapred.TaskTracker.<init>(TaskTracker.java:1108)
         at org.apache.hadoop.mapred.TaskTracker.main(TaskTracker.java: 
2985)

I've fixed that in the past but I can't remember what I did, searching  
through mailing lists only points to different jars in the classpath.   
There's only one copy of the hadoop*jar files on the classpath.  Also  
going to the :50030/jobtracker.jsp page gives back a 500 error saying  
that it can't find the org.apache.hadoop.mapred.ClusterStatus class  
which is in hadoop-mapred-0.21.0-SNAPSHOT.jar 

Mime
View raw message