Ma= ybe the OP was running into something else, will wait for him to return wit= h more information.

Thanks,

+Vinod

O= n Tue, Dec 20, 2011 at 11:29 AM, Robert Evans <evans@yahoo-inc.com> wrote:

Should we file a JIRA so that the MapReduce AM blows up in a more obv= ious fashion if Shuffle is not configured?

--Bobby Evans

On 12/20/11 12:30 PM, "Vinod Kumar Vavilapalli" <vinodkv@hortonworks.com= > wrote:

I guess you don't have shuffle configured. Can you look at the applicat= ion master (AM) logs and paste logs from there? There will a link to AM log= s on the application page of RM web UI.

You can also check and see if shuffle is configured. From the INSTALL file = (http://svn.apache.org/repos/asf/ha= doop/common/trunk/hadoop-mapreduce-project/INSTALL),

Step 7) Setup confi= g: for running mapreduce applications, which now are in user land, you need= to setup nodemanager with the following configuration in your yarn-site.xm= l before you start the nodemanager.
=A0 =A0 <property>
=A0 =A0 =A0 <name>yarn.nodemanager.aux-services</name>
=A0 =A0 =A0 <value>mapreduce.shuffle</value>
=A0 =A0 </property>

=A0 =A0 <property>
=A0 =A0 =A0 <name>yarn.nodemanager.aux-services.mapreduce.shuffle.cla= ss</name>
=A0 =A0 =A0 <value>org.apache.hadoop.mapred.ShuffleHandler</value&= gt;
=A0 =A0 </property>

Step 8) Modify mapred-site.xml to use yarn framework
=A0 =A0 <property> =A0 =A0
=A0 =A0 =A0 <name> mapreduce.framework.name <http://mapreduce.framework.name> <= ;/name>
=A0 =A0 =A0 <value>yarn</value> =A0
=A0 =A0 </property>

+Vinod

On Tue, Dec 20, 2011 at 8:12 AM, Arun C Murthy <acm@hortonworks.com> wrote:

Can you look at the= /nodes web-page to see how many nodes you have?

Also, do you see any exceptions in the ResourceManager logs on dn5?

Arun

On Dec 20, 2011, at 5:14 AM, Jingui Lee wrote:

Hi,all

I am running hadoop 0.23 on 5 nodes.

I could run any YARN application or Mapreduce Job on this cluster before.
But, after I changed Resourcemanager Node from node4 to node5, when I run a= pplications (I have modified property referenced in configure file), map an= d reduce process will hang up at 0% until I killed the application.

I don't know why.

terminal output:

bin/hadoop jar hadoop-mapreduce-examples-0.23.0.jar wordcount /share/stdinp= ut/1k /testread/hao
11/12/20 20:20:29 INFO mapreduce.Cluster: Cannot pick org.apache.hadoop.map= red.LocalClientProtocolProvider as the ClientProtocolProvider - returned nu= ll protocol
11/12/20 20:20:29 INFO ipc.YarnRPC: Creating YarnRPC for org.apache.hadoop.= yarn.ipc.HadoopYarnProtoRPC
11/12/20 20:20:29 INFO mapred.ResourceMgrDelegate: Connecting to ResourceMa= nager at dn5/192.1= 68.3.204:50010 <http://192.168.3.204:50010/>

11/12/20 20:20:29 INFO ipc.HadoopYarnRPC: Creating a HadoopYarnProtoRpc pro= xy for protocol interface org.apache.hadoop.yarn.api.ClientRMProtocol
11/12/20 20:20:29 INFO mapred.ResourceMgrDelegate: Connected to ResourceMan= ager at dn5/192.16= 8.3.204:50010 <http://192.168.3.204:50010/>
11/12/20 20:20:29 WARN conf.Configuration: fs.default.name <http://fs.default.name/> =A0is deprecated. I= nstead, use fs.defaultFS

11/12/20 20:20:29 WARN conf.Configuration: mapred.used.genericoptionsparser= is deprecated. Instead, use mapreduce.client.genericoptionsparser.used
11/12/20 20:20:29 INFO input.FileInputFormat: Total input paths to process = : 1
11/12/20 20:20:29 INFO util.NativeCodeLoader: Loaded the native-hadoop libr= ary
11/12/20 20:20:29 WARN snappy.LoadSnappy: Snappy native library not loaded<= br> 11/12/20 20:20:29 INFO mapreduce.JobSubmitter: number of splits:1
11/12/20 20:20:29 INFO mapred.YARNRunner: AppMaster capability =3D memory: = 2048
11/12/20 20:20:29 INFO mapred.YARNRunner: Command to launch container for A= pplicationMaster is : $JAVA_HOME/bin/java -Dlog4j.configuration=3Dcontainer= -log4j.properties -Dyarn.app.mapreduce.container.log.dir=3D<LOG_DIR> = -Dyarn.app.mapreduce.container.log.filesize=3D0 -Dhadoop.root.logger=3DINFO= ,CLA -Xmx1536m org.apache.hadoop.mapreduce.v2.app.MRAppMaster 1><LOG_= DIR>/stdout 2><LOG_DIR>/stderr
11/12/20 20:20:29 INFO mapred.ResourceMgrDelegate: Submitted application ap= plication_1324372145692_0004 to ResourceManager
11/12/20 20:20:29 INFO mapred.ClientCache: Connecting to HistoryServer at: = dn5:10020
11/12/20 20:20:29 INFO ipc.YarnRPC: Creating YarnRPC for org.apache.hadoop.= yarn.ipc.HadoopYarnProtoRPC
11/12/20 20:20:29 INFO mapred.ClientCache: Connected to HistoryServer at: d= n5:10020
11/12/20 20:20:29 INFO ipc.HadoopYarnRPC: Creating a HadoopYarnProtoRpc pro= xy for protocol interface org.apache.hadoop.mapreduce.v2.api.MRClientProtoc= ol
11/12/20 20:20:30 INFO mapreduce.Job: Running job: job_1324372145692_0004 11/12/20 20:20:31 INFO mapreduce.Job:=A0 map 0% reduce 0%