hadoop-mapreduce-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Pratik Gadiya <pratik_gad...@persistent.com>
Subject Apache Hadoop 2.2 - Job execution error
Date Fri, 02 Jan 2015 14:37:33 GMT
Hi All,

I am using Pivotal Distribution to deploy Hadoop cluster.

For the same, I am following link mentioned below
http://pivotalhd.docs.pivotal.io/doc/2100/webhelp/topics/ManuallyInstallingandUsingPivotalHD21Stack.html


Deployment Configuration :

1) phd1.xyz.com - NameNode, ResourceManager
2) phd2.xyz.com - DataNode, NodeManager

I have above mentioned services UP and Running and also able to access the HDFS file system
but not able to execute jobs on cluster

Stderr file log :
Error: Could not find or load main class org.apache.hadoop.mapreduce.v2.app.MRAppMaster

In order to resolve the same problem, I added classpath property tag in yarn-site.xml but
still I am not able to execute the job, resulting into same error as above.

yarn-site.xml :
<property>
<name>yarn.application.classpath</name>
<value>/usr/lib/gphd/hadoop/*,/usr/lib/gphd/hadoop/lib/*,/usr/lib/gphd/hadoop-hdfs/*,/usr/lib/gphd/hadoop-hdfs/lib/*,/usr/lib/gphd/hadoop-yarn/*,/usr/lib/gphd/hadoop-yarn/lib/*,/usr/lib/gphd/hadoop-mapreduce/*,/usr/lib/gphd/hadoop-mapreduce/lib/*,/etc/gphd/hadoop/conf/*</value>
</property>


[root@phd2 bin]# ls /usr/lib/gphd/hadoop/
bin                                        hadoop-annotations.jar                hadoop-common-2.2.0-gphd-3.1.0.0-tests.jar
 hadoop-nfs.jar  NOTICE.txt
CHANGES.txt                                hadoop-auth-2.2.0-gphd-3.1.0.0.jar    hadoop-common.jar
                          lib             README.txt
etc                                        hadoop-auth.jar                       hadoop-common-tests.jar
                    libexec         sbin
hadoop-annotations-2.2.0-gphd-3.1.0.0.jar  hadoop-common-2.2.0-gphd-3.1.0.0.jar  hadoop-nfs-2.2.0-gphd-3.1.0.0.jar
          LICENSE.txt


[root@phd2 bin]# ls /usr/lib/gphd/hadoop/lib
activation-1.1.jar                commons-httpclient-3.1.jar    jasper-runtime-5.5.23.jar
               jetty-security-7.6.10.v20130312.jar  native
asm-3.2.jar                       commons-io-2.1.jar            javax.servlet-2.5.0.v201103041518.jar
   jetty-server-7.6.10.v20130312.jar    netty-3.6.2.Final.jar
avro-1.7.4.jar                    commons-lang-2.5.jar          jaxb-api-2.2.2.jar       
               jetty-servlet-7.6.10.v20130312.jar   paranamer-2.3.jar
commons-beanutils-1.7.0.jar       commons-logging-1.1.1.jar     jaxb-impl-2.2.3-1.jar    
               jetty-util-7.6.10.v20130312.jar      protobuf-java-2.5.0.jar
commons-beanutils-core-1.8.0.jar  commons-math-2.1.jar          jersey-core-1.9.jar      
               jetty-webapp-7.6.10.v20130312.jar    servlet-api-2.5.jar
commons-cli-1.2.jar               commons-net-3.1.jar           jersey-json-1.9.jar      
               jetty-xml-7.6.10.v20130312.jar       slf4j-api-1.7.5.jar
commons-codec-1.4.jar             guava-11.0.2.jar              jersey-server-1.9.jar    
               jsch-0.1.42.jar                      slf4j-log4j12-1.7.5.jar
commons-collections-3.2.1.jar     jackson-core-asl-1.8.8.jar    jets3t-0.6.1.jar         
               jsp-api-2.1.jar                      snappy-java-1.0.4.1.jar
commons-compress-1.4.1.jar        jackson-jaxrs-1.8.8.jar       jettison-1.1.jar         
               jsr305-1.3.9.jar                     stax-api-1.0.1.jar
commons-configuration-1.6.jar     jackson-mapper-asl-1.8.8.jar  jetty-continuation-7.6.10.v20130312.jar
 junit-4.8.2.jar                      xmlenc-0.52.jar
commons-digester-1.8.jar          jackson-xc-1.8.8.jar          jetty-http-7.6.10.v20130312.jar
         log4j-1.2.17.jar                     xz-1.0.jar
commons-el-1.0.jar                jasper-compiler-5.5.23.jar    jetty-io-7.6.10.v20130312.jar
           mockito-all-1.8.5.jar                zookeeper.jar


[root@phd2 bin]# ls /usr/lib/gphd/hadoop-yarn/lib
aopalliance-1.0.jar         commons-io-2.1.jar     jackson-core-asl-1.8.8.jar    jersey-guice-1.9.jar
  netty-3.6.2.Final.jar    xz-1.0.jar
asm-3.2.jar                 guice-3.0.jar          jackson-mapper-asl-1.8.8.jar  jersey-server-1.9.jar
 paranamer-2.3.jar
avro-1.7.4.jar              guice-servlet-3.0.jar  javax.inject-1.jar            junit-4.10.jar
        protobuf-java-2.5.0.jar
commons-compress-1.4.1.jar  hamcrest-core-1.1.jar  jersey-core-1.9.jar           log4j-1.2.17.jar
      snappy-java-1.0.4.1.jar


I am not sure which all environment varibles should be exported or described in the hadoop-env.sh
or othere configuration files.

Note :
Pivotal version which I have installed supports Apache Hadoop 2.2

Remember:
While replying, please keep gmail id in loop

Let me know how can I resolve the error.


With Regards,
Pratik


DISCLAIMER
==========
This e-mail may contain privileged and confidential information which is the property of Persistent
Systems Ltd. It is intended only for the use of the individual or entity to which it is addressed.
If you are not the intended recipient, you are not authorized to read, retain, copy, print,
distribute or use this message. If you have received this communication in error, please notify
the sender and delete all copies of this message. Persistent Systems Ltd. does not accept
any liability for virus infected mails.


Mime
View raw message