hadoop-mapreduce-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Mahadev Konar <maha...@hortonworks.com>
Subject Re: Can't find main class when run ResourceManager or NodeManager
Date Thu, 11 Aug 2011 21:07:41 GMT
Matei,
 Are you sure you are using the latest common from trunk?  The ClientCache was recently added
within the last few weeks.
  Looks like its using some old version of hadoop common.

 Also the sl4j errors should be fixed in the latest MR-279 branch. 

thanks
mahadev

On Aug 11, 2011, at 1:52 PM, Matei Zaharia wrote:

> I get a similar error on Mac OS X. I've built YARN and extracted the tarball to a directory,
but when I run bin/yarn-daemon.sh start nodemanager, it prints the following to its log (apologies
for the long trace):
> 
> log4j:WARN No appenders could be found for logger (org.apache.hadoop.metrics2.impl.MetricsSystemImpl).
> log4j:WARN Please initialize the log4j system properly.
> log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for more info.
> Exception in thread "main" org.apache.hadoop.yarn.YarnException: Failed to Start org.apache.hadoop.yarn.server.nodemanager.NodeManager
> 	at org.apache.hadoop.yarn.service.CompositeService.start(CompositeService.java:80)
> 	at org.apache.hadoop.yarn.server.nodemanager.NodeManager.start(NodeManager.java:146)
> 	at org.apache.hadoop.yarn.server.nodemanager.NodeManager.main(NodeManager.java:191)
> Caused by: org.apache.avro.AvroRuntimeException: org.apache.hadoop.yarn.YarnException:
java.lang.reflect.InvocationTargetException
> 	at org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl.start(NodeStatusUpdaterImpl.java:140)
> 	at org.apache.hadoop.yarn.service.CompositeService.start(CompositeService.java:68)
> 	... 2 more
> Caused by: org.apache.hadoop.yarn.YarnException: java.lang.reflect.InvocationTargetException
> 	at org.apache.hadoop.yarn.factories.impl.pb.RpcClientFactoryPBImpl.getClient(RpcClientFactoryPBImpl.java:70)
> 	at org.apache.hadoop.yarn.ipc.HadoopYarnProtoRPC.getProxy(HadoopYarnProtoRPC.java:35)
> 	at org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl.getRMClient(NodeStatusUpdaterImpl.java:158)
> 	at org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl.registerWithRM(NodeStatusUpdaterImpl.java:163)
> 	at org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl.start(NodeStatusUpdaterImpl.java:136)
> 	... 3 more
> Caused by: java.lang.reflect.InvocationTargetException
> 	at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
> 	at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:39)
> 	at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:27)
> 	at java.lang.reflect.Constructor.newInstance(Constructor.java:513)
> 	at org.apache.hadoop.yarn.factories.impl.pb.RpcClientFactoryPBImpl.getClient(RpcClientFactoryPBImpl.java:67)
> 	... 7 more
> Caused by: java.lang.NoClassDefFoundError: org/apache/hadoop/ipc/ClientCache
> 	at org.apache.hadoop.yarn.ipc.ProtoOverHadoopRpcEngine.<clinit>(ProtoOverHadoopRpcEngine.java:63)
> 	at java.lang.Class.forName0(Native Method)
> 	at java.lang.Class.forName(Class.java:247)
> 	at org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:1108)
> 	at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:1160)
> 	at org.apache.hadoop.ipc.RPC.getProtocolEngine(RPC.java:94)
> 	at org.apache.hadoop.ipc.RPC.getProtocolProxy(RPC.java:422)
> 	at org.apache.hadoop.ipc.RPC.getProtocolProxy(RPC.java:368)
> 	at org.apache.hadoop.ipc.RPC.getProtocolProxy(RPC.java:333)
> 	at org.apache.hadoop.ipc.RPC.getProtocolProxy(RPC.java:461)
> 	at org.apache.hadoop.ipc.RPC.getProxy(RPC.java:442)
> 	at org.apache.hadoop.yarn.server.api.impl.pb.client.ResourceTrackerPBClientImpl.<init>(ResourceTrackerPBClientImpl.java:32)
> 	... 12 more
> Caused by: java.lang.ClassNotFoundException: org.apache.hadoop.ipc.ClientCache
> 	at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
> 	at java.security.AccessController.doPrivileged(Native Method)
> 	at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
> 	at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
> 	at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
> 	at java.lang.ClassLoader.loadClass(ClassLoader.java:247)
> 	... 24 more
> 
> In addition, if I try to start resourcemanager instead, it logs:
> 
> SLF4J: Class path contains multiple SLF4J bindings.
> SLF4J: Found binding in [jar:file:/Users/matei/workspace/MR-279/common/build/ivy/lib/Hadoop-Common/common/slf4j-log4j12-1.6.1.jar!/org/slf4j/impl/StaticLoggerBinder.class]
> SLF4J: Found binding in [jar:file:/Users/matei/workspace/MR-279-deploy/hadoop-mapreduce-1.0-SNAPSHOT/lib/slf4j-log4j12-1.6.1.jar!/org/slf4j/impl/StaticLoggerBinder.class]
> SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
> log4j:WARN No appenders could be found for logger (org.apache.hadoop.metrics2.lib.MutableMetricsFactory).
> log4j:WARN Please initialize the log4j system properly.
> log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for more info.
> Exception in thread "main" java.lang.IllegalStateException: For this operation, current
State must be STARTED instead of INITED
>        at org.apache.hadoop.yarn.service.AbstractService.ensureCurrentState(AbstractService.java:101)
>        at org.apache.hadoop.yarn.service.AbstractService.stop(AbstractService.java:69)
>        at org.apache.hadoop.yarn.server.resourcemanager.amlauncher.ApplicationMasterLauncher.stop(ApplicationMasterLauncher.java:90)
>        at org.apache.hadoop.yarn.service.CompositeService.stop(CompositeService.java:89)
>        at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.stop(ResourceManager.java:423)
>        at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.main(ResourceManager.java:493)
> 
> I don't know where the multiple SLF4J versions are coming from because I checked out
common, hdfs and mapreduce at the same time.
> 
> Matei
> 
> 
> 
> 
> On Aug 11, 2011, at 2:01 AM, Vinod KV wrote:
> 
>> 
>> 
>> Report YARN_* and HADOOP_* variables that you've set before starting the daemons.
>> 
>> Also run 'bin/yarn classpath' and post the output of that command too. If the output
doesn't contain yarn-server-resourcemanager-1.0-SNAPSHOT.jar, you are missing something.
>> 
>> +Vinod
>> 
>> 
>> On Thursday 11 August 2011 01:57 AM, Haoyuan Li wrote:
>>> Hi,
>>> 
>>> When I ran ResourceManager or NodeManager as steps here
>>> http://svn.apache.org/repos/asf/hadoop/common/branches/MR-279/mapreduce/INSTALL
>>> .
>>> It has main class can't be found exception... I attached the shell output
>>> here. Any help will be appreciated!
>>> 
>>> Thanks,
>>> 
>>> Haoyuan
>>> 
>>> haoyuan@hya:~/hadoop/hadoop-mapreduce-1.0-SNAPSHOT$ ./bin/yarn-daemon.sh
>>> start resourcemanager
>>> starting resourcemanager, logging to
>>> /home/haoyuan/hadoop/hadoop-mapreduce-1.0-SNAPSHOT/bin/../logs/yarn-haoyuan-resourcemanager-hya.out
>>> /usr/lib/jvm/java-6-sun/bin/java -Dproc_resourcemanager -Xmx1000m
>>> -Dhadoop.log.dir=/home/haoyuan/hadoop/hadoop-mapreduce-1.0-SNAPSHOT/bin/../logs
>>> -Dyarn.log.dir=/home/haoyuan/hadoop/hadoop-mapreduce-1.0-SNAPSHOT/bin/../logs
>>> -Dhadoop.log.file=yarn-haoyuan-resourcemanager-hya.log
>>> -Dyarn.log.file=yarn-haoyuan-resourcemanager-hya.log -Dyarn.home.dir=
>>> -Dyarn.id.str=haoyuan -Dhadoop.root.logger=INFO,DRFA
>>> -Dyarn.root.logger=INFO,DRFA -Dyarn.policy.file=hadoop-policy.xml
>>> -Dhadoop.log.dir=/home/haoyuan/hadoop/hadoop-mapreduce-1.0-SNAPSHOT/bin/../logs
>>> -Dyarn.log.dir=/home/haoyuan/hadoop/hadoop-mapreduce-1.0-SNAPSHOT/bin/../logs
>>> -Dhadoop.log.file=yarn-haoyuan-resourcemanager-hya.log
>>> -Dyarn.log.file=yarn-haoyuan-resourcemanager-hya.log
>>> -Dyarn.home.dir=/home/haoyuan/hadoop/hadoop-mapreduce-1.0-SNAPSHOT/bin/..
>>> -Dhadoop.root.logger=INFO,DRFA -Dyarn.root.logger=INFO,DRFA -classpath
>>> /home/haoyuan/hadoop/conf:/home/haoyuan/hadoop/conf:/usr/lib/jvm/java-6-sun/lib/tools.jar:/home/haoyuan/hadoop/trunk/common/hadoop-*.jar:/home/haoyuan/hadoop/trunk/common/lib/*.jar:/home/haoyuan/hadoop/trunk/common/share/hadoop/common/*.jar:/home/haoyuan/hadoop/trunk/common/share/hadoop/common/lib/*.jar:/home/haoyuan/hadoop/trunk/common/share/hadoop/hdfs/*.jar:/home/haoyuan/hadoop/trunk/hdfs/hadoop-*.jar:/home/haoyuan/hadoop/trunk/hdfs/lib/*.jar:/home/haoyuan/hadoop/trunk/hdfs/build/classes:/home/haoyuan/hadoop/trunk/hdfs/hadoop-*.jar:/home/haoyuan/hadoop/trunk/hdfs/lib/*.jar:/home/haoyuan/hadoop/trunk/mapreduce/build/classes:/home/haoyuan/hadoop/trunk/mapreduce/build:/home/haoyuan/hadoop/trunk/mapreduce/build/test/classes:/home/haoyuan/hadoop/trunk/mapreduce/build/tools:/home/haoyuan/hadoop/trunk/mapreduce/lib/*.jar:/home/haoyuan/hadoop/trunk/mapreduce/*.jar:/home/haoyuan/hadoop/hadoop-mapreduce-1.0-SNAPSHOT/bin/../lib/aopalliance-1.0.jar:/home/haoyuan/hadoop/hadoop-mapreduce-1.0-SNAPSHOT/bin/../lib/asm-3.2.jar:/home/haoyuan/hadoop/hadoop-mapreduce-1.0-SNAPSHOT/bin/../lib/aspectjrt-1.6.5.jar:/home/haoyuan/hadoop/hadoop-mapreduce-1.0-SNAPSHOT/bin/../lib/avro-1.3.2.jar:/home/haoyuan/hadoop/hadoop-mapreduce-1.0-SNAPSHOT/bin/../lib/avro-1.4.1.jar:/home/haoyuan/hadoop/hadoop-mapreduce-1.0-SNAPSHOT/bin/../lib/clover-3.0.2.jar:/home/haoyuan/hadoop/hadoop-mapreduce-1.0-SNAPSHOT/bin/../lib/commons-beanutils-1.7.0.jar:/home/haoyuan/hadoop/hadoop-mapreduce-1.0-SNAPSHOT/bin/../lib/commons-beanutils-core-1.8.0.jar:/home/haoyuan/hadoop/hadoop-mapreduce-1.0-SNAPSHOT/bin/../lib/commons-cli-1.2.jar:/home/haoyuan/hadoop/hadoop-mapreduce-1.0-SNAPSHOT/bin/../lib/commons-codec-1.4.jar:/home/haoyuan/hadoop/hadoop-mapreduce-1.0-SNAPSHOT/bin/../lib/commons-collections-3.2.1.jar:/home/haoyuan/hadoop/hadoop-mapreduce-1.0-SNAPSHOT/bin/../lib/commons-configuration-1.6.jar:/home/haoyuan/hadoop/hadoop-mapreduce-1.0-SNAPSHOT/bin/../lib/commons-digester-1.8.jar:/home/haoyuan/hadoop/hadoop-mapreduce-1.0-SNAPSHOT/bin/../lib/commons-httpclient-3.1.jar:/home/haoyuan/hadoop/hadoop-mapreduce-1.0-SNAPSHOT/bin/../lib/commons-lang-2.5.jar:/home/haoyuan/hadoop/hadoop-mapreduce-1.0-SNAPSHOT/bin/../lib/commons-logging-1.0.4.jar:/home/haoyuan/hadoop/hadoop-mapreduce-1.0-SNAPSHOT/bin/../lib/commons-logging-api-1.0.4.jar:/home/haoyuan/hadoop/hadoop-mapreduce-1.0-SNAPSHOT/bin/../lib/commons-math-2.1.jar:/home/haoyuan/hadoop/hadoop-mapreduce-1.0-SNAPSHOT/bin/../lib/commons-net-1.4.1.jar:/home/haoyuan/hadoop/hadoop-mapreduce-1.0-SNAPSHOT/bin/../lib/core-3.1.1.jar:/home/haoyuan/hadoop/hadoop-mapreduce-1.0-SNAPSHOT/bin/../lib/guava-r09.jar:/home/haoyuan/hadoop/hadoop-mapreduce-1.0-SNAPSHOT/bin/../lib/guice-2.0.jar:/home/haoyuan/hadoop/hadoop-mapreduce-1.0-SNAPSHOT/bin/../lib/guice-servlet-2.0.jar:/home/haoyuan/hadoop/hadoop-mapreduce-1.0-SNAPSHOT/bin/../lib/hadoop-annotations-0.23.0-SNAPSHOT.jar:/home/haoyuan/hadoop/hadoop-mapreduce-1.0-SNAPSHOT/bin/../lib/jackson-core-asl-1.4.2.jar:/home/haoyuan/hadoop/hadoop-mapreduce-1.0-SNAPSHOT/bin/../lib/jackson-mapper-asl-1.4.2.jar:/home/haoyuan/hadoop/hadoop-mapreduce-1.0-SNAPSHOT/bin/../lib/jdiff-1.0.9.jar:/home/haoyuan/hadoop/hadoop-mapreduce-1.0-SNAPSHOT/bin/../lib/jets3t-0.6.1.jar:/home/haoyuan/hadoop/hadoop-mapreduce-1.0-SNAPSHOT/bin/../lib/jetty-6.1.26.jar:/home/haoyuan/hadoop/hadoop-mapreduce-1.0-SNAPSHOT/bin/../lib/jetty-util-6.1.26.jar:/home/haoyuan/hadoop/hadoop-mapreduce-1.0-SNAPSHOT/bin/../lib/jsp-api-2.1.jar:/home/haoyuan/hadoop/hadoop-mapreduce-1.0-SNAPSHOT/bin/../lib/junit-4.8.2.jar:/home/haoyuan/hadoop/hadoop-mapreduce-1.0-SNAPSHOT/bin/../lib/kfs-0.3.jar:/home/haoyuan/hadoop/hadoop-mapreduce-1.0-SNAPSHOT/bin/../lib/log4j-1.2.15.jar:/home/haoyuan/hadoop/hadoop-mapreduce-1.0-SNAPSHOT/bin/../lib/netty-3.2.3.Final.jar:/home/haoyuan/hadoop/hadoop-mapreduce-1.0-SNAPSHOT/bin/../lib/oro-2.0.8.jar:/home/haoyuan/hadoop/hadoop-mapreduce-1.0-SNAPSHOT/bin/../lib/paranamer-2.2.jar:/home/haoyuan/hadoop/hadoop-mapreduce-1.0-SNAPSHOT/bin/../lib/paranamer-ant-2.2.jar:/home/haoyuan/hadoop/hadoop-mapreduce-1.0-SNAPSHOT/bin/../lib/paranamer-generator-2.2.jar:/home/haoyuan/hadoop/hadoop-mapreduce-1.0-SNAPSHOT/bin/../lib/protobuf-java-2.4.0a.jar:/home/haoyuan/hadoop/hadoop-mapreduce-1.0-SNAPSHOT/bin/../lib/qdox-1.10.1.jar:/home/haoyuan/hadoop/hadoop-mapreduce-1.0-SNAPSHOT/bin/../lib/servlet-api-2.5.jar:/home/haoyuan/hadoop/hadoop-mapreduce-1.0-SNAPSHOT/bin/../lib/slf4j-api-1.6.1.jar:/home/haoyuan/hadoop/hadoop-mapreduce-1.0-SNAPSHOT/bin/../lib/slf4j-log4j12-1.6.1.jar:/home/haoyuan/hadoop/hadoop-mapreduce-1.0-SNAPSHOT/bin/../lib/xmlenc-0.52.jar:/home/haoyuan/hadoop/hadoop-mapreduce-1.0-SNAPSHOT/bin/../modules/hadoop-mapreduce-client-app-1.0-SNAPSHOT.jar:/home/haoyuan/hadoop/hadoop-mapreduce-1.0-SNAPSHOT/bin/../modules/hadoop-mapreduce-client-common-1.0-SNAPSHOT.jar:/home/haoyuan/hadoop/hadoop-mapreduce-1.0-SNAPSHOT/bin/../modules/hadoop-mapreduce-client-core-1.0-SNAPSHOT.jar:/home/haoyuan/hadoop/hadoop-mapreduce-1.0-SNAPSHOT/bin/../modules/hadoop-mapreduce-client-hs-1.0-SNAPSHOT.jar:/home/haoyuan/hadoop/hadoop-mapreduce-1.0-SNAPSHOT/bin/../modules/hadoop-mapreduce-client-jobclient-1.0-SNAPSHOT.jar:/home/haoyuan/hadoop/hadoop-mapreduce-1.0-SNAPSHOT/bin/../modules/hadoop-mapreduce-client-shuffle-1.0-SNAPSHOT.jar:/home/haoyuan/hadoop/hadoop-mapreduce-1.0-SNAPSHOT/bin/../modules/yarn-api-1.0-SNAPSHOT.jar:/home/haoyuan/hadoop/hadoop-mapreduce-1.0-SNAPSHOT/bin/../modules/yarn-common-1.0-SNAPSHOT.jar:/home/haoyuan/hadoop/hadoop-mapreduce-1.0-SNAPSHOT/bin/../modules/yarn-server-common-1.0-SNAPSHOT.jar:/home/haoyuan/hadoop/hadoop-mapreduce-1.0-SNAPSHOT/bin/../modules/yarn-server-nodemanager-1.0-SNAPSHOT.jar:/home/haoyuan/hadoop/hadoop-mapreduce-1.0-SNAPSHOT/bin/../modules/yarn-server-resourcemanager-1.0-SNAPSHOT.jar:/home/haoyuan/hadoop/conf/rm-config/log4j.properties
>>> org.apache.hadoop.yarn.server.resourcemanager.ResourceManager
>>> Exception in thread "main" java.lang.NoClassDefFoundError:
>>> org/apache/hadoop/conf/Configuration
>>> Caused by: java.lang.ClassNotFoundException:
>>> org.apache.hadoop.conf.Configuration
>>> at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
>>> at java.security.AccessController.doPrivileged(Native Method)
>>> at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
>>> at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
>>> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
>>> at java.lang.ClassLoader.loadClass(ClassLoader.java:247)
>>> Could not find the main class:
>>> org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.  Program will
>>> exit.
>>> 
>> 
>> 
> 


Mime
View raw message