hadoop-mapreduce-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Paul Rogers <paul.roge...@gmail.com>
Subject Issue with map reduce on examples
Date Mon, 18 Nov 2013 12:53:36 GMT
Hi All

Having some problems with map reduce running in pseudo-distributed mode.  I
am running version 1.2.1 on linux.  I have:
1. created $JAVA_HOME & $HADOOP_HOME and added the relative bin directories
to the path;
2. Formatted the dfs;
3. executed start-dfs.sh and start-mapred.sh.

Executing jps seems to show everything running that should be running (I
think).

[paul@lt001 bin]$ jps
8724 TaskTracker
8487 SecondaryNameNode
8841 Jps
8353 DataNode
7239 NameNode
8597 JobTracker

I have then tried to run the wordcount and pi examples with similar
results, eg:

[paul@lt001 bin]$ hadoop jar hadoop/hadoop-examples-1.2.1.jar pi 4 1000
Warning: $HADOOP_HOME is deprecated.

Number of Maps  = 4
Samples per Map = 1000
Wrote input for Map #0
Wrote input for Map #1
Wrote input for Map #2
Wrote input for Map #3
Starting Job
13/11/18 10:31:38 INFO mapred.FileInputFormat: Total input paths to process
: 4
13/11/18 10:31:39 INFO mapred.JobClient: Running job: job_201311181028_0001
13/11/18 10:31:40 INFO mapred.JobClient:  map 0% reduce 0%
13/11/18 10:31:47 INFO mapred.JobClient:  map 50% reduce 0%
13/11/18 10:31:52 INFO mapred.JobClient:  map 100% reduce 0%

In each instance the output reaches the map 100% reduce 0% stage then
stalls.  No matter how long I wait the job does not advance any further.  I
have checked the logs and the one I suspect is indicating the problem
is hadoop-paul-tasktracker-lt001.log which has the following output:

2013-11-18 10:31:55,969 INFO org.apache.hadoop.mapred.TaskTracker:
attempt_201311181028_0001_r_000000_0 0.0% reduce > copy >
2013-11-18 10:34:59,148 INFO org.apache.hadoop.mapred.TaskTracker:
attempt_201311181028_0001_r_000000_0 0.0% reduce > copy >
2013-11-18 10:35:05,196 INFO org.apache.hadoop.mapred.TaskTracker:
attempt_201311181028_0001_r_000000_0 0.0% reduce > copy >
2013-11-18 10:35:11,253 INFO org.apache.hadoop.mapred.TaskTracker:
attempt_201311181028_0001_r_000000_0 0.0% reduce > copy >

..........

2013-11-18 11:10:03,259 INFO org.apache.hadoop.mapred.TaskTracker:
attempt_201311181028_0001_r_000000_0 0.0% reduce > copy >
2013-11-18 11:10:06,290 INFO org.apache.hadoop.mapred.TaskTracker:
attempt_201311181028_0001_r_000000_0 0.0% reduce > copy >
2013-11-18 11:10:12,320 INFO org.apache.hadoop.mapred.TaskTracker:
attempt_201311181028_0001_r_000000_0 0.0% reduce > copy >
2013-11-18 11:10:18,343 INFO org.apache.hadoop.mapred.TaskTracker:
attempt_201311181028_0001_r_000000_0 0.0% reduce > copy >
2013-11-18 11:10:21,369 INFO org.apache.hadoop.mapred.TaskTracker:
attempt_201311181028_0001_r_000000_0 0.0% reduce > copy >
2013-11-18 11:10:27,395 INFO org.apache.hadoop.mapred.TaskTracker:
attempt_201311181028_0001_r_000000_0 0.0% reduce > copy >
2013-11-18 11:10:33,426 INFO org.apache.hadoop.mapred.TaskTracker:
attempt_201311181028_0001_r_000000_0 0.0% reduce > copy >
2013-11-18 11:10:36,463 INFO org.apache.hadoop.mapred.TaskTracker:
attempt_201311181028_0001_r_000000_0 0.0% reduce > copy >

It seems it is stuck on reduce > copy > but why?  Can anyone help with
where to look next?

Many thanks


Paul

Mime
View raw message