hadoop-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ted Yu <yuzhih...@gmail.com>
Subject Re: Apache Hadoop 2.2 - Job execution error
Date Fri, 02 Jan 2015 14:50:28 GMT
Have you asked this question on Pivotal's mailing list ?

Cheers



> On Jan 2, 2015, at 6:37 AM, Pratik Gadiya <pratik_gadiya@persistent.com> wrote:
> 
> Hi All,
>  
> I am using Pivotal Distribution to deploy Hadoop cluster.
>  
> For the same, I am following link mentioned below
> http://pivotalhd.docs.pivotal.io/doc/2100/webhelp/topics/ManuallyInstallingandUsingPivotalHD21Stack.html
>  
>  
> Deployment Configuration :
>  
> 1) phd1.xyz.com - NameNode, ResourceManager
> 2) phd2.xyz.com - DataNode, NodeManager
>  
> I have above mentioned services UP and Running and also able to access the HDFS file
system but not able to execute jobs on cluster
>  
> Stderr file log :
> Error: Could not find or load main class org.apache.hadoop.mapreduce.v2.app.MRAppMaster
>  
> In order to resolve the same problem, I added classpath property tag in yarn-site.xml
but still I am not able to execute the job, resulting into same error as above.
>  
> yarn-site.xml :
> <property>
> <name>yarn.application.classpath</name>
> <value>/usr/lib/gphd/hadoop/*,/usr/lib/gphd/hadoop/lib/*,/usr/lib/gphd/hadoop-hdfs/*,/usr/lib/gphd/hadoop-hdfs/lib/*,/usr/lib/gphd/hadoop-yarn/*,/usr/lib/gphd/hadoop-yarn/lib/*,/usr/lib/gphd/hadoop-mapreduce/*,/usr/lib/gphd/hadoop-mapreduce/lib/*,/etc/gphd/hadoop/conf/*</value>
> </property>
>  
>  
> [root@phd2 bin]# ls /usr/lib/gphd/hadoop/
> bin                                        hadoop-annotations.jar                hadoop-common-2.2.0-gphd-3.1.0.0-tests.jar
 hadoop-nfs.jar  NOTICE.txt
> CHANGES.txt                                hadoop-auth-2.2.0-gphd-3.1.0.0.jar    hadoop-common.jar
                          lib             README.txt
> etc                                        hadoop-auth.jar                       hadoop-common-tests.jar
                    libexec         sbin
> hadoop-annotations-2.2.0-gphd-3.1.0.0.jar  hadoop-common-2.2.0-gphd-3.1.0.0.jar  hadoop-nfs-2.2.0-gphd-3.1.0.0.jar
          LICENSE.txt
>  
>  
> [root@phd2 bin]# ls /usr/lib/gphd/hadoop/lib
> activation-1.1.jar                commons-httpclient-3.1.jar    jasper-runtime-5.5.23.jar
               jetty-security-7.6.10.v20130312.jar  native
> asm-3.2.jar                       commons-io-2.1.jar            javax.servlet-2.5.0.v201103041518.jar
   jetty-server-7.6.10.v20130312.jar    netty-3.6.2.Final.jar
> avro-1.7.4.jar                    commons-lang-2.5.jar          jaxb-api-2.2.2.jar  
                    jetty-servlet-7.6.10.v20130312.jar   paranamer-2.3.jar
> commons-beanutils-1.7.0.jar       commons-logging-1.1.1.jar     jaxb-impl-2.2.3-1.jar
                   jetty-util-7.6.10.v20130312.jar      protobuf-java-2.5.0.jar
> commons-beanutils-core-1.8.0.jar  commons-math-2.1.jar          jersey-core-1.9.jar 
                    jetty-webapp-7.6.10.v20130312.jar    servlet-api-2.5.jar
> commons-cli-1.2.jar               commons-net-3.1.jar           jersey-json-1.9.jar 
                    jetty-xml-7.6.10.v20130312.jar       slf4j-api-1.7.5.jar
> commons-codec-1.4.jar             guava-11.0.2.jar              jersey-server-1.9.jar
                   jsch-0.1.42.jar                      slf4j-log4j12-1.7.5.jar
> commons-collections-3.2.1.jar     jackson-core-asl-1.8.8.jar    jets3t-0.6.1.jar    
                    jsp-api-2.1.jar                      snappy-java-1.0.4.1.jar
> commons-compress-1.4.1.jar        jackson-jaxrs-1.8.8.jar       jettison-1.1.jar    
                    jsr305-1.3.9.jar                     stax-api-1.0.1.jar
> commons-configuration-1.6.jar     jackson-mapper-asl-1.8.8.jar  jetty-continuation-7.6.10.v20130312.jar
 junit-4.8.2.jar                      xmlenc-0.52.jar
> commons-digester-1.8.jar          jackson-xc-1.8.8.jar          jetty-http-7.6.10.v20130312.jar
         log4j-1.2.17.jar                     xz-1.0.jar
> commons-el-1.0.jar                jasper-compiler-5.5.23.jar    jetty-io-7.6.10.v20130312.jar
           mockito-all-1.8.5.jar                zookeeper.jar
>  
>  
> [root@phd2 bin]# ls /usr/lib/gphd/hadoop-yarn/lib
> aopalliance-1.0.jar         commons-io-2.1.jar     jackson-core-asl-1.8.8.jar    jersey-guice-1.9.jar
  netty-3.6.2.Final.jar    xz-1.0.jar
> asm-3.2.jar                 guice-3.0.jar          jackson-mapper-asl-1.8.8.jar  jersey-server-1.9.jar
 paranamer-2.3.jar
> avro-1.7.4.jar              guice-servlet-3.0.jar  javax.inject-1.jar            junit-4.10.jar
        protobuf-java-2.5.0.jar
> commons-compress-1.4.1.jar  hamcrest-core-1.1.jar  jersey-core-1.9.jar           log4j-1.2.17.jar
      snappy-java-1.0.4.1.jar
>  
>  
> I am not sure which all environment varibles should be exported or described in the hadoop-env.sh
or othere configuration files.
>  
> Note :
> Pivotal version which I have installed supports Apache Hadoop 2.2
>  
> Remember:
> While replying, please keep gmail id in loop
>  
> Let me know how can I resolve the error.
>  
>  
> With Regards,
> Pratik
>  
> DISCLAIMER ========== This e-mail may contain privileged and confidential information
which is the property of Persistent Systems Ltd. It is intended only for the use of the individual
or entity to which it is addressed. If you are not the intended recipient, you are not authorized
to read, retain, copy, print, distribute or use this message. If you have received this communication
in error, please notify the sender and delete all copies of this message. Persistent Systems
Ltd. does not accept any liability for virus infected mails.

Mime
View raw message