Return-Path: X-Original-To: apmail-hadoop-mapreduce-dev-archive@minotaur.apache.org Delivered-To: apmail-hadoop-mapreduce-dev-archive@minotaur.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 7AB3574F9 for ; Thu, 11 Aug 2011 20:53:05 +0000 (UTC) Received: (qmail 33323 invoked by uid 500); 11 Aug 2011 20:53:04 -0000 Delivered-To: apmail-hadoop-mapreduce-dev-archive@hadoop.apache.org Received: (qmail 33261 invoked by uid 500); 11 Aug 2011 20:53:04 -0000 Mailing-List: contact mapreduce-dev-help@hadoop.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: mapreduce-dev@hadoop.apache.org Delivered-To: mailing list mapreduce-dev@hadoop.apache.org Received: (qmail 33252 invoked by uid 99); 11 Aug 2011 20:53:03 -0000 Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 11 Aug 2011 20:53:03 +0000 X-ASF-Spam-Status: No, hits=-1.3 required=5.0 tests=RCVD_IN_DNSWL_MED,SPF_SOFTFAIL X-Spam-Check-By: apache.org Received-SPF: softfail (athena.apache.org: transitioning domain of matei@eecs.berkeley.edu does not designate 169.229.218.142 as permitted sender) Received: from [169.229.218.142] (HELO cm01fe.IST.Berkeley.EDU) (169.229.218.142) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 11 Aug 2011 20:52:57 +0000 Received: from dhcp-44-88.eecs.berkeley.edu ([128.32.44.88]) by cm01fe.ist.berkeley.edu with esmtpsa (TLSv1:AES128-SHA:128) (Exim 4.76) (auth plain:matei@eecs.berkeley.edu) (envelope-from ) id 1QrcF1-0006mR-3X; Thu, 11 Aug 2011 13:52:36 -0700 Subject: Re: Can't find main class when run ResourceManager or NodeManager Mime-Version: 1.0 (Apple Message framework v1084) Content-Type: text/plain; charset=us-ascii From: Matei Zaharia In-Reply-To: <4E439A5A.1070904@yahoo-inc.com> Date: Thu, 11 Aug 2011 13:52:34 -0700 Cc: Haoyuan Li Content-Transfer-Encoding: quoted-printable Message-Id: References: <4E439A5A.1070904@yahoo-inc.com> To: mapreduce-dev@hadoop.apache.org X-Mailer: Apple Mail (2.1084) I get a similar error on Mac OS X. I've built YARN and extracted the = tarball to a directory, but when I run bin/yarn-daemon.sh start = nodemanager, it prints the following to its log (apologies for the long = trace): log4j:WARN No appenders could be found for logger = (org.apache.hadoop.metrics2.impl.MetricsSystemImpl). log4j:WARN Please initialize the log4j system properly. log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for = more info. Exception in thread "main" org.apache.hadoop.yarn.YarnException: Failed = to Start org.apache.hadoop.yarn.server.nodemanager.NodeManager at = org.apache.hadoop.yarn.service.CompositeService.start(CompositeService.jav= a:80) at = org.apache.hadoop.yarn.server.nodemanager.NodeManager.start(NodeManager.ja= va:146) at = org.apache.hadoop.yarn.server.nodemanager.NodeManager.main(NodeManager.jav= a:191) Caused by: org.apache.avro.AvroRuntimeException: = org.apache.hadoop.yarn.YarnException: = java.lang.reflect.InvocationTargetException at = org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl.start(Node= StatusUpdaterImpl.java:140) at = org.apache.hadoop.yarn.service.CompositeService.start(CompositeService.jav= a:68) ... 2 more Caused by: org.apache.hadoop.yarn.YarnException: = java.lang.reflect.InvocationTargetException at = org.apache.hadoop.yarn.factories.impl.pb.RpcClientFactoryPBImpl.getClient(= RpcClientFactoryPBImpl.java:70) at = org.apache.hadoop.yarn.ipc.HadoopYarnProtoRPC.getProxy(HadoopYarnProtoRPC.= java:35) at = org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl.getRMClien= t(NodeStatusUpdaterImpl.java:158) at = org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl.registerWi= thRM(NodeStatusUpdaterImpl.java:163) at = org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl.start(Node= StatusUpdaterImpl.java:136) ... 3 more Caused by: java.lang.reflect.InvocationTargetException at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native = Method) at = sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAcc= essorImpl.java:39) at = sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstr= uctorAccessorImpl.java:27) at = java.lang.reflect.Constructor.newInstance(Constructor.java:513) at = org.apache.hadoop.yarn.factories.impl.pb.RpcClientFactoryPBImpl.getClient(= RpcClientFactoryPBImpl.java:67) ... 7 more Caused by: java.lang.NoClassDefFoundError: = org/apache/hadoop/ipc/ClientCache at = org.apache.hadoop.yarn.ipc.ProtoOverHadoopRpcEngine.(ProtoOverHado= opRpcEngine.java:63) at java.lang.Class.forName0(Native Method) at java.lang.Class.forName(Class.java:247) at = org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:110= 8) at = org.apache.hadoop.conf.Configuration.getClass(Configuration.java:1160) at org.apache.hadoop.ipc.RPC.getProtocolEngine(RPC.java:94) at org.apache.hadoop.ipc.RPC.getProtocolProxy(RPC.java:422) at org.apache.hadoop.ipc.RPC.getProtocolProxy(RPC.java:368) at org.apache.hadoop.ipc.RPC.getProtocolProxy(RPC.java:333) at org.apache.hadoop.ipc.RPC.getProtocolProxy(RPC.java:461) at org.apache.hadoop.ipc.RPC.getProxy(RPC.java:442) at = org.apache.hadoop.yarn.server.api.impl.pb.client.ResourceTrackerPBClientIm= pl.(ResourceTrackerPBClientImpl.java:32) ... 12 more Caused by: java.lang.ClassNotFoundException: = org.apache.hadoop.ipc.ClientCache at java.net.URLClassLoader$1.run(URLClassLoader.java:202) at java.security.AccessController.doPrivileged(Native Method) at java.net.URLClassLoader.findClass(URLClassLoader.java:190) at java.lang.ClassLoader.loadClass(ClassLoader.java:306) at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301) at java.lang.ClassLoader.loadClass(ClassLoader.java:247) ... 24 more In addition, if I try to start resourcemanager instead, it logs: SLF4J: Class path contains multiple SLF4J bindings. SLF4J: Found binding in = [jar:file:/Users/matei/workspace/MR-279/common/build/ivy/lib/Hadoop-Common= /common/slf4j-log4j12-1.6.1.jar!/org/slf4j/impl/StaticLoggerBinder.class] SLF4J: Found binding in = [jar:file:/Users/matei/workspace/MR-279-deploy/hadoop-mapreduce-1.0-SNAPSH= OT/lib/slf4j-log4j12-1.6.1.jar!/org/slf4j/impl/StaticLoggerBinder.class] SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an = explanation. log4j:WARN No appenders could be found for logger = (org.apache.hadoop.metrics2.lib.MutableMetricsFactory). log4j:WARN Please initialize the log4j system properly. log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for = more info. Exception in thread "main" java.lang.IllegalStateException: For this = operation, current State must be STARTED instead of INITED at = org.apache.hadoop.yarn.service.AbstractService.ensureCurrentState(Abstract= Service.java:101) at = org.apache.hadoop.yarn.service.AbstractService.stop(AbstractService.java:6= 9) at = org.apache.hadoop.yarn.server.resourcemanager.amlauncher.ApplicationMaster= Launcher.stop(ApplicationMasterLauncher.java:90) at = org.apache.hadoop.yarn.service.CompositeService.stop(CompositeService.java= :89) at = org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.stop(Resourc= eManager.java:423) at = org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.main(Resourc= eManager.java:493) I don't know where the multiple SLF4J versions are coming from because I = checked out common, hdfs and mapreduce at the same time. Matei On Aug 11, 2011, at 2:01 AM, Vinod KV wrote: >=20 >=20 > Report YARN_* and HADOOP_* variables that you've set before starting = the daemons. >=20 > Also run 'bin/yarn classpath' and post the output of that command too. = If the output doesn't contain = yarn-server-resourcemanager-1.0-SNAPSHOT.jar, you are missing something. >=20 > +Vinod >=20 >=20 > On Thursday 11 August 2011 01:57 AM, Haoyuan Li wrote: >> Hi, >>=20 >> When I ran ResourceManager or NodeManager as steps here >> = http://svn.apache.org/repos/asf/hadoop/common/branches/MR-279/mapreduce/IN= STALL >> . >> It has main class can't be found exception... I attached the shell = output >> here. Any help will be appreciated! >>=20 >> Thanks, >>=20 >> Haoyuan >>=20 >> haoyuan@hya:~/hadoop/hadoop-mapreduce-1.0-SNAPSHOT$ = ./bin/yarn-daemon.sh >> start resourcemanager >> starting resourcemanager, logging to >> = /home/haoyuan/hadoop/hadoop-mapreduce-1.0-SNAPSHOT/bin/../logs/yarn-haoyua= n-resourcemanager-hya.out >> /usr/lib/jvm/java-6-sun/bin/java -Dproc_resourcemanager -Xmx1000m >> = -Dhadoop.log.dir=3D/home/haoyuan/hadoop/hadoop-mapreduce-1.0-SNAPSHOT/bin/= ../logs >> = -Dyarn.log.dir=3D/home/haoyuan/hadoop/hadoop-mapreduce-1.0-SNAPSHOT/bin/..= /logs >> -Dhadoop.log.file=3Dyarn-haoyuan-resourcemanager-hya.log >> -Dyarn.log.file=3Dyarn-haoyuan-resourcemanager-hya.log = -Dyarn.home.dir=3D >> -Dyarn.id.str=3Dhaoyuan -Dhadoop.root.logger=3DINFO,DRFA >> -Dyarn.root.logger=3DINFO,DRFA -Dyarn.policy.file=3Dhadoop-policy.xml >> = -Dhadoop.log.dir=3D/home/haoyuan/hadoop/hadoop-mapreduce-1.0-SNAPSHOT/bin/= ../logs >> = -Dyarn.log.dir=3D/home/haoyuan/hadoop/hadoop-mapreduce-1.0-SNAPSHOT/bin/..= /logs >> -Dhadoop.log.file=3Dyarn-haoyuan-resourcemanager-hya.log >> -Dyarn.log.file=3Dyarn-haoyuan-resourcemanager-hya.log >> = -Dyarn.home.dir=3D/home/haoyuan/hadoop/hadoop-mapreduce-1.0-SNAPSHOT/bin/.= . >> -Dhadoop.root.logger=3DINFO,DRFA -Dyarn.root.logger=3DINFO,DRFA = -classpath >> = /home/haoyuan/hadoop/conf:/home/haoyuan/hadoop/conf:/usr/lib/jvm/java-6-su= n/lib/tools.jar:/home/haoyuan/hadoop/trunk/common/hadoop-*.jar:/home/haoyu= an/hadoop/trunk/common/lib/*.jar:/home/haoyuan/hadoop/trunk/common/share/h= adoop/common/*.jar:/home/haoyuan/hadoop/trunk/common/share/hadoop/common/l= ib/*.jar:/home/haoyuan/hadoop/trunk/common/share/hadoop/hdfs/*.jar:/home/h= aoyuan/hadoop/trunk/hdfs/hadoop-*.jar:/home/haoyuan/hadoop/trunk/hdfs/lib/= *.jar:/home/haoyuan/hadoop/trunk/hdfs/build/classes:/home/haoyuan/hadoop/t= runk/hdfs/hadoop-*.jar:/home/haoyuan/hadoop/trunk/hdfs/lib/*.jar:/home/hao= yuan/hadoop/trunk/mapreduce/build/classes:/home/haoyuan/hadoop/trunk/mapre= duce/build:/home/haoyuan/hadoop/trunk/mapreduce/build/test/classes:/home/h= aoyuan/hadoop/trunk/mapreduce/build/tools:/home/haoyuan/hadoop/trunk/mapre= duce/lib/*.jar:/home/haoyuan/hadoop/trunk/mapreduce/*.jar:/home/haoyuan/ha= doop/hadoop-mapreduce-1.0-SNAPSHOT/bin/../lib/aopalliance-1.0.jar:/home/ha= oyuan/hadoop/hadoop-mapreduce-1.0-SNAPSHOT/bin/../lib/asm-3.2.jar:/home/ha= oyuan/hadoop/hadoop-mapreduce-1.0-SNAPSHOT/bin/../lib/aspectjrt-1.6.5.jar:= /home/haoyuan/hadoop/hadoop-mapreduce-1.0-SNAPSHOT/bin/../lib/avro-1.3.2.j= ar:/home/haoyuan/hadoop/hadoop-mapreduce-1.0-SNAPSHOT/bin/../lib/avro-1.4.= 1.jar:/home/haoyuan/hadoop/hadoop-mapreduce-1.0-SNAPSHOT/bin/../lib/clover= -3.0.2.jar:/home/haoyuan/hadoop/hadoop-mapreduce-1.0-SNAPSHOT/bin/../lib/c= ommons-beanutils-1.7.0.jar:/home/haoyuan/hadoop/hadoop-mapreduce-1.0-SNAPS= HOT/bin/../lib/commons-beanutils-core-1.8.0.jar:/home/haoyuan/hadoop/hadoo= p-mapreduce-1.0-SNAPSHOT/bin/../lib/commons-cli-1.2.jar:/home/haoyuan/hado= op/hadoop-mapreduce-1.0-SNAPSHOT/bin/../lib/commons-codec-1.4.jar:/home/ha= oyuan/hadoop/hadoop-mapreduce-1.0-SNAPSHOT/bin/../lib/commons-collections-= 3.2.1.jar:/home/haoyuan/hadoop/hadoop-mapreduce-1.0-SNAPSHOT/bin/../lib/co= mmons-configuration-1.6.jar:/home/haoyuan/hadoop/hadoop-mapreduce-1.0-SNAP= SHOT/bin/../lib/commons-digester-1.8.jar:/home/haoyuan/hadoop/hadoop-mapre= duce-1.0-SNAPSHOT/bin/../lib/commons-httpclient-3.1.jar:/home/haoyuan/hado= op/hadoop-mapreduce-1.0-SNAPSHOT/bin/../lib/commons-lang-2.5.jar:/home/hao= yuan/hadoop/hadoop-mapreduce-1.0-SNAPSHOT/bin/../lib/commons-logging-1.0.4= .jar:/home/haoyuan/hadoop/hadoop-mapreduce-1.0-SNAPSHOT/bin/../lib/commons= -logging-api-1.0.4.jar:/home/haoyuan/hadoop/hadoop-mapreduce-1.0-SNAPSHOT/= bin/../lib/commons-math-2.1.jar:/home/haoyuan/hadoop/hadoop-mapreduce-1.0-= SNAPSHOT/bin/../lib/commons-net-1.4.1.jar:/home/haoyuan/hadoop/hadoop-mapr= educe-1.0-SNAPSHOT/bin/../lib/core-3.1.1.jar:/home/haoyuan/hadoop/hadoop-m= apreduce-1.0-SNAPSHOT/bin/../lib/guava-r09.jar:/home/haoyuan/hadoop/hadoop= -mapreduce-1.0-SNAPSHOT/bin/../lib/guice-2.0.jar:/home/haoyuan/hadoop/hado= op-mapreduce-1.0-SNAPSHOT/bin/../lib/guice-servlet-2.0.jar:/home/haoyuan/h= adoop/hadoop-mapreduce-1.0-SNAPSHOT/bin/../lib/hadoop-annotations-0.23.0-S= NAPSHOT.jar:/home/haoyuan/hadoop/hadoop-mapreduce-1.0-SNAPSHOT/bin/../lib/= jackson-core-asl-1.4.2.jar:/home/haoyuan/hadoop/hadoop-mapreduce-1.0-SNAPS= HOT/bin/../lib/jackson-mapper-asl-1.4.2.jar:/home/haoyuan/hadoop/hadoop-ma= preduce-1.0-SNAPSHOT/bin/../lib/jdiff-1.0.9.jar:/home/haoyuan/hadoop/hadoo= p-mapreduce-1.0-SNAPSHOT/bin/../lib/jets3t-0.6.1.jar:/home/haoyuan/hadoop/= hadoop-mapreduce-1.0-SNAPSHOT/bin/../lib/jetty-6.1.26.jar:/home/haoyuan/ha= doop/hadoop-mapreduce-1.0-SNAPSHOT/bin/../lib/jetty-util-6.1.26.jar:/home/= haoyuan/hadoop/hadoop-mapreduce-1.0-SNAPSHOT/bin/../lib/jsp-api-2.1.jar:/h= ome/haoyuan/hadoop/hadoop-mapreduce-1.0-SNAPSHOT/bin/../lib/junit-4.8.2.ja= r:/home/haoyuan/hadoop/hadoop-mapreduce-1.0-SNAPSHOT/bin/../lib/kfs-0.3.ja= r:/home/haoyuan/hadoop/hadoop-mapreduce-1.0-SNAPSHOT/bin/../lib/log4j-1.2.= 15.jar:/home/haoyuan/hadoop/hadoop-mapreduce-1.0-SNAPSHOT/bin/../lib/netty= -3.2.3.Final.jar:/home/haoyuan/hadoop/hadoop-mapreduce-1.0-SNAPSHOT/bin/..= /lib/oro-2.0.8.jar:/home/haoyuan/hadoop/hadoop-mapreduce-1.0-SNAPSHOT/bin/= ../lib/paranamer-2.2.jar:/home/haoyuan/hadoop/hadoop-mapreduce-1.0-SNAPSHO= T/bin/../lib/paranamer-ant-2.2.jar:/home/haoyuan/hadoop/hadoop-mapreduce-1= .0-SNAPSHOT/bin/../lib/paranamer-generator-2.2.jar:/home/haoyuan/hadoop/ha= doop-mapreduce-1.0-SNAPSHOT/bin/../lib/protobuf-java-2.4.0a.jar:/home/haoy= uan/hadoop/hadoop-mapreduce-1.0-SNAPSHOT/bin/../lib/qdox-1.10.1.jar:/home/= haoyuan/hadoop/hadoop-mapreduce-1.0-SNAPSHOT/bin/../lib/servlet-api-2.5.ja= r:/home/haoyuan/hadoop/hadoop-mapreduce-1.0-SNAPSHOT/bin/../lib/slf4j-api-= 1.6.1.jar:/home/haoyuan/hadoop/hadoop-mapreduce-1.0-SNAPSHOT/bin/../lib/sl= f4j-log4j12-1.6.1.jar:/home/haoyuan/hadoop/hadoop-mapreduce-1.0-SNAPSHOT/b= in/../lib/xmlenc-0.52.jar:/home/haoyuan/hadoop/hadoop-mapreduce-1.0-SNAPSH= OT/bin/../modules/hadoop-mapreduce-client-app-1.0-SNAPSHOT.jar:/home/haoyu= an/hadoop/hadoop-mapreduce-1.0-SNAPSHOT/bin/../modules/hadoop-mapreduce-cl= ient-common-1.0-SNAPSHOT.jar:/home/haoyuan/hadoop/hadoop-mapreduce-1.0-SNA= PSHOT/bin/../modules/hadoop-mapreduce-client-core-1.0-SNAPSHOT.jar:/home/h= aoyuan/hadoop/hadoop-mapreduce-1.0-SNAPSHOT/bin/../modules/hadoop-mapreduc= e-client-hs-1.0-SNAPSHOT.jar:/home/haoyuan/hadoop/hadoop-mapreduce-1.0-SNA= PSHOT/bin/../modules/hadoop-mapreduce-client-jobclient-1.0-SNAPSHOT.jar:/h= ome/haoyuan/hadoop/hadoop-mapreduce-1.0-SNAPSHOT/bin/../modules/hadoop-map= reduce-client-shuffle-1.0-SNAPSHOT.jar:/home/haoyuan/hadoop/hadoop-mapredu= ce-1.0-SNAPSHOT/bin/../modules/yarn-api-1.0-SNAPSHOT.jar:/home/haoyuan/had= oop/hadoop-mapreduce-1.0-SNAPSHOT/bin/../modules/yarn-common-1.0-SNAPSHOT.= jar:/home/haoyuan/hadoop/hadoop-mapreduce-1.0-SNAPSHOT/bin/../modules/yarn= -server-common-1.0-SNAPSHOT.jar:/home/haoyuan/hadoop/hadoop-mapreduce-1.0-= SNAPSHOT/bin/../modules/yarn-server-nodemanager-1.0-SNAPSHOT.jar:/home/hao= yuan/hadoop/hadoop-mapreduce-1.0-SNAPSHOT/bin/../modules/yarn-server-resou= rcemanager-1.0-SNAPSHOT.jar:/home/haoyuan/hadoop/conf/rm-config/log4j.prop= erties >> org.apache.hadoop.yarn.server.resourcemanager.ResourceManager >> Exception in thread "main" java.lang.NoClassDefFoundError: >> org/apache/hadoop/conf/Configuration >> Caused by: java.lang.ClassNotFoundException: >> org.apache.hadoop.conf.Configuration >> at java.net.URLClassLoader$1.run(URLClassLoader.java:202) >> at java.security.AccessController.doPrivileged(Native Method) >> at java.net.URLClassLoader.findClass(URLClassLoader.java:190) >> at java.lang.ClassLoader.loadClass(ClassLoader.java:306) >> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301) >> at java.lang.ClassLoader.loadClass(ClassLoader.java:247) >> Could not find the main class: >> org.apache.hadoop.yarn.server.resourcemanager.ResourceManager. = Program will >> exit. >>=20 >=20 >=20