hadoop-mapreduce-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Arun C Murthy <...@hortonworks.com>
Subject Re: Map and Reduce process hang out at 0%
Date Tue, 20 Dec 2011 16:12:23 GMT
Can you look at the /nodes web-page to see how many nodes you have?

Also, do you see any exceptions in the ResourceManager logs on dn5?

Arun

On Dec 20, 2011, at 5:14 AM, Jingui Lee wrote:

> Hi,all
> 
> I am running hadoop 0.23 on 5 nodes.
> 
> I could run any YARN application or Mapreduce Job on this cluster before.
> 
> But, after I changed Resourcemanager Node from node4 to node5, when I run applications
(I have modified property referenced in configure file), map and reduce process will hang
up at 0% until I killed the application.
> 
> I don't know why.
> 
> terminal output:
> 
> bin/hadoop jar hadoop-mapreduce-examples-0.23.0.jar wordcount /share/stdinput/1k /testread/hao
> 11/12/20 20:20:29 INFO mapreduce.Cluster: Cannot pick org.apache.hadoop.mapred.LocalClientProtocolProvider
as the ClientProtocolProvider - returned null protocol
> 11/12/20 20:20:29 INFO ipc.YarnRPC: Creating YarnRPC for org.apache.hadoop.yarn.ipc.HadoopYarnProtoRPC
> 11/12/20 20:20:29 INFO mapred.ResourceMgrDelegate: Connecting to ResourceManager at dn5/192.168.3.204:50010
> 11/12/20 20:20:29 INFO ipc.HadoopYarnRPC: Creating a HadoopYarnProtoRpc proxy for protocol
interface org.apache.hadoop.yarn.api.ClientRMProtocol
> 11/12/20 20:20:29 INFO mapred.ResourceMgrDelegate: Connected to ResourceManager at dn5/192.168.3.204:50010
> 11/12/20 20:20:29 WARN conf.Configuration: fs.default.name is deprecated. Instead, use
fs.defaultFS
> 11/12/20 20:20:29 WARN conf.Configuration: mapred.used.genericoptionsparser is deprecated.
Instead, use mapreduce.client.genericoptionsparser.used
> 11/12/20 20:20:29 INFO input.FileInputFormat: Total input paths to process : 1
> 11/12/20 20:20:29 INFO util.NativeCodeLoader: Loaded the native-hadoop library
> 11/12/20 20:20:29 WARN snappy.LoadSnappy: Snappy native library not loaded
> 11/12/20 20:20:29 INFO mapreduce.JobSubmitter: number of splits:1
> 11/12/20 20:20:29 INFO mapred.YARNRunner: AppMaster capability = memory: 2048
> 11/12/20 20:20:29 INFO mapred.YARNRunner: Command to launch container for ApplicationMaster
is : $JAVA_HOME/bin/java -Dlog4j.configuration=container-log4j.properties -Dyarn.app.mapreduce.container.log.dir=<LOG_DIR>
-Dyarn.app.mapreduce.container.log.filesize=0 -Dhadoop.root.logger=INFO,CLA -Xmx1536m org.apache.hadoop.mapreduce.v2.app.MRAppMaster
1><LOG_DIR>/stdout 2><LOG_DIR>/stderr 
> 11/12/20 20:20:29 INFO mapred.ResourceMgrDelegate: Submitted application application_1324372145692_0004
to ResourceManager
> 11/12/20 20:20:29 INFO mapred.ClientCache: Connecting to HistoryServer at: dn5:10020
> 11/12/20 20:20:29 INFO ipc.YarnRPC: Creating YarnRPC for org.apache.hadoop.yarn.ipc.HadoopYarnProtoRPC
> 11/12/20 20:20:29 INFO mapred.ClientCache: Connected to HistoryServer at: dn5:10020
> 11/12/20 20:20:29 INFO ipc.HadoopYarnRPC: Creating a HadoopYarnProtoRpc proxy for protocol
interface org.apache.hadoop.mapreduce.v2.api.MRClientProtocol
> 11/12/20 20:20:30 INFO mapreduce.Job: Running job: job_1324372145692_0004
> 11/12/20 20:20:31 INFO mapreduce.Job:  map 0% reduce 0%
> 
> 


Mime
View raw message