Return-Path: X-Original-To: apmail-hadoop-mapreduce-user-archive@minotaur.apache.org Delivered-To: apmail-hadoop-mapreduce-user-archive@minotaur.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 1D3E5C10D for ; Tue, 5 Jun 2012 10:26:04 +0000 (UTC) Received: (qmail 85596 invoked by uid 500); 5 Jun 2012 10:26:02 -0000 Delivered-To: apmail-hadoop-mapreduce-user-archive@hadoop.apache.org Received: (qmail 85429 invoked by uid 500); 5 Jun 2012 10:26:02 -0000 Mailing-List: contact mapreduce-user-help@hadoop.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: mapreduce-user@hadoop.apache.org Delivered-To: mailing list mapreduce-user@hadoop.apache.org Received: (qmail 85415 invoked by uid 99); 5 Jun 2012 10:26:02 -0000 Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 05 Jun 2012 10:26:02 +0000 X-ASF-Spam-Status: No, hits=2.2 required=5.0 tests=HTML_MESSAGE,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (athena.apache.org: local policy) Received: from [212.122.43.157] (HELO datameer.com) (212.122.43.157) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 05 Jun 2012 10:25:57 +0000 Received: from [192.168.178.170] (unknown [192.168.178.170]) by datameer.com (Postfix) with ESMTPSA id 5269B1099FEF for ; Tue, 5 Jun 2012 12:25:35 +0200 (CEST) From: Subroto Mime-Version: 1.0 (Apple Message framework v1278) Content-Type: multipart/alternative; boundary="Apple-Mail=_86963538-DA78-4C76-BAD3-F60A0B1E68FE" Subject: Re: java.lang.NoClassDefFoundError: org/apache/hadoop/mapreduce/v2/app/MRAppMaste Date: Tue, 5 Jun 2012 12:25:34 +0200 In-Reply-To: <06006DDA5A27D541991944AC4117E7A95A9FD46A@szxeml524-mbx.china.huawei.com> To: mapreduce-user@hadoop.apache.org References: <7214F671-C78B-4073-8411-B88D2EADA0AC@datameer.com> <06006DDA5A27D541991944AC4117E7A95A9FD389@szxeml524-mbx.china.huawei.com> <06006DDA5A27D541991944AC4117E7A95A9FD46A@szxeml524-mbx.china.huawei.com> Message-Id: <2A771854-8496-431F-A186-DCCFF61CA5FB@datameer.com> X-Mailer: Apple Mail (2.1278) X-Virus-Checked: Checked by ClamAV on apache.org --Apple-Mail=_86963538-DA78-4C76-BAD3-F60A0B1E68FE Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=iso-8859-1 Hi Deva, Tried the yarn application path with absolute values. Still it didn't = work. It failed with same stack trace:-( Now the value of yarn.application.classpath was: = /usr/local/hadoop/etc/hadoop,/usr/local/hadoop/*,/usr/local/hadoop/lib/*,/= usr/local/hadoop/*,/usr/local/hadoop/lib/*,/usr/local/hadoop/*,/usr/local/= hadoop/lib/*,/usr/local/hadoop/*,/usr/local/hadoop/* Cheers, Subroto Sanyal On Jun 5, 2012, at 12:07 PM, Devaraj k wrote: > Hi Subroto, > =20 > It will not use yarn-env.sh for launching the application master. = NM uses the environment set by the client for launching application = master. Can you set the environment variables in /etc/profile or update = the yarn application classpath with the absolute paths. > =20 > =20 > Thanks > Devaraj > From: Subroto [ssanyal@datameer.com] > Sent: Tuesday, June 05, 2012 2:25 PM > To: mapreduce-user@hadoop.apache.org > Subject: Re: java.lang.NoClassDefFoundError: = org/apache/hadoop/mapreduce/v2/app/MRAppMaste >=20 > Hi Deva, >=20 > Thanks for your response.=20 > The file etc/hadoop/yarn-env.sh has the following entries: > export HADOOP_MAPRED_HOME=3D/usr/local/hadoop > export HADOOP_COMMON_HOME=3D/usr/local/hadoop > export HADOOP_HDFS_HOME=3D/usr/local/hadoop > export YARN_HOME=3D/usr/local/hadoop > export HADOOP_CONF_DIR=3D/usr/local/hadoop/etc/hadoop > export YARN_CONF_DIR=3D$HADOOP_CONF_DIR >=20 >=20 > Is it expected to have these variables in profile file of the Linux = user?? >=20 > I am not using Windows client. My client is running on Mac and the = cluster is running on Linux versions. >=20 > Cheers, > Subroto Sanyal > On Jun 5, 2012, at 10:50 AM, Devaraj k wrote: >=20 >> Can you check all the hadoop environment variables are set properly = in which the app master is getting launching. >> =20 >> =20 >> If you are submitting from windows, this might be the issue = https://issues.apache.org/jira/browse/MAPREDUCE-4052. >> =20 >> Thanks >> Devaraj >> From: Subroto [ssanyal@datameer.com] >> Sent: Tuesday, June 05, 2012 2:14 PM >> To: mapreduce-user@hadoop.apache.org >> Subject: java.lang.NoClassDefFoundError: = org/apache/hadoop/mapreduce/v2/app/MRAppMaste >>=20 >> Hi, >>=20 >> While running MR Jobs over a yarn cluster I keep on getting: >> Exception in thread "main" java.lang.NoClassDefFoundError: = org/apache/hadoop/mapreduce/v2/app/MRAppMaster >> Caused by: java.lang.ClassNotFoundException: = org.apache.hadoop.mapreduce.v2.app.MRAppMaster >> at java.net.URLClassLoader$1.run(URLClassLoader.java:202) >> at java.security.AccessController.doPrivileged(Native Method) >> at java.net.URLClassLoader.findClass(URLClassLoader.java:190) >> at java.lang.ClassLoader.loadClass(ClassLoader.java:307) >> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301) >> at java.lang.ClassLoader.loadClass(ClassLoader.java:248) >> Could not find the main class: = org.apache.hadoop.mapreduce.v2.app.MRAppMaster. Program will exit. >>=20 >> My client is running from a different environment from where the = cluster is running. >> If I submit a job from the cluster environment; it runs successfully. >>=20 >> I have verified the property yarn.application.classpath before = submitting it from the client. The value is set to: >> = $HADOOP_CONF_DIR,$HADOOP_COMMON_HOME/*,$HADOOP_COMMON_HOME/lib/*,$HADOOP_H= DFS_HOME/*,$HADOOP_HDFS_HOME/lib/*,$HADOOP_MAPRED_HOME/*,$HADOOP_MAPRED_HO= ME/lib/*,$YARN_HOME/*,$YARN_HOME/lib/* >>=20 >> Please let me know if I am missing anything. >>=20 >> Cheers, >> Subroto Sanyal --Apple-Mail=_86963538-DA78-4C76-BAD3-F60A0B1E68FE Content-Transfer-Encoding: quoted-printable Content-Type: text/html; charset=iso-8859-1 Hi Deva,

Tried the yarn = application path with absolute values. Still it didn't work.  It = failed with same stack trace:-(
Now the value of yarn.application.classpath was:
 

From: Subroto [ssanyal@datameer.com]
Sent:=  Tuesday, June 05, = 2012 2:25 PM
To: mapreduce-user@hadoop.apa= che.org
Subject: Re: = java.lang.NoClassDefFoundError: = org/apache/hadoop/mapreduce/v2/app/MRAppMaste

Hi Deva,

Thanks for your = response. 
The file etc/hadoop/yarn-env.sh has the = following entries:
export = HADOOP_MAPRED_HOME=3D/usr/local/hadoop
export = HADOOP_COMMON_HOME=3D/usr/local/hadoop
export = HADOOP_HDFS_HOME=3D/usr/local/hadoop
export = YARN_HOME=3D/usr/local/hadoop
export = HADOOP_CONF_DIR=3D/usr/local/hadoop/etc/hadoop
export = YARN_CONF_DIR=3D$HADOOP_CONF_DIR


=
Is it expected to have these variables in profile file of the Linux = user??

I am not using Windows client. My client = is running on Mac and the cluster is running on Linux = versions.

Cheers,
Subroto = Sanyal
On Jun 5, 2012, at 10:50 AM, Devaraj k = wrote:

 

 

If you are submitting = from windows, this might be the issue https://issues.apache.org/jira/browse/MAPREDUCE-4052= .

 

Thanks
Devaraj
From: Subroto [ssanyal@datameer.com]
Sent: Tuesday, June 05, 2012 2:14 = PM
To: mapreduce-user@hadoop.apache.org
Subject:<= span = class=3D"Apple-converted-space"> java.lang.NoClassDefFoundErro= r: = org/apache/hadoop/mapreduce/v2/app/MRAppMaste

Hi,

While running MR Jobs over a yarn cluster = I  keep on getting:
Exception in thread "main" =
java.lang.NoClassDefFoundError: =
org/apache/hadoop/mapreduce/v2/app/MRAppMaster
Caused by: java.lang.ClassNotFoundException: =
org.apache.hadoop.mapreduce.v2.app.MRAppMaster
	at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
	at java.security.AccessController.doPrivileged(Native Method)
	at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
	at java.lang.ClassLoader.loadClass(ClassLoader.java:307)
	at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
	at java.lang.ClassLoader.loadClass(ClassLoader.java:248)
Could not find the main class: =
org.apache.hadoop.mapreduce.v2.app.MRAppMaster.  Program will =
exit.

My client is running from a = different environment from where the cluster is running.
If I = submit a job from the cluster environment; it runs = successfully.

I have verified the = property yarn.application.classpath before submitting it = from the client. The value is set to:
lib/*
Please let me know if I am missing = anything.