hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From CTS-RAAJ <rajesh.moha...@cognizant.com>
Subject Failed to get reducer output while running in cluster
Date Fri, 04 Jul 2008 12:51:23 GMT

Hi All,

After all successful work aroud in getting the hadoop cluster run, I am
kicked back by some problem while executig my program in cluster,
Program takes input a log file, I have written a mapper which goes through
the line to find certain pattern and pass the mapper output to reducer to
sort and give me the counters. This I pass to a small job, which creates a
chart using apache poi and itext api.

When I execute locally it runs fine with follwoing output

LOCAL
08/07/04 17:24:23 INFO mapred.LocalJobRunner:
file:/home/rajesh/POC/hadoop-0.17.0/sdeinput1/SDE_LOG.log:0+27131
08/07/04 17:24:23 INFO mapred.TaskRunner: Task 'job_local_1_map_0000' done.
08/07/04 17:24:23 INFO mapred.TaskRunner: Saved output of task
'job_local_1_map_0000' to file:/home/rajesh/POC/hadoop-0.17.0/output1
08/07/04 17:24:23 INFO mapred.LocalJobRunner: reduce > reduce
08/07/04 17:24:23 INFO mapred.TaskRunner: Task 'reduce_75lmh0' done.
08/07/04 17:24:23 INFO mapred.TaskRunner: Saved output of task
'reduce_75lmh0' to file:/home/rajesh/POC/hadoop-0.17.0/output108/07/04
17:24:23 INFO mapred.JobClient: Job complete: job_local_1
08/07/04 17:24:23 INFO mapred.JobClient: Counters: 11
08/07/04 17:24:23 INFO mapred.JobClient:   Map-Reduce Framework
08/07/04 17:24:23 INFO mapred.JobClient:     Map input records=316
08/07/04 17:24:23 INFO mapred.JobClient:     Map output records=102
08/07/04 17:24:23 INFO mapred.JobClient:     Map input bytes=27131
08/07/04 17:24:23 INFO mapred.JobClient:     Map output bytes=816
08/07/04 17:24:23 INFO mapred.JobClient:     Combine input records=102
08/07/04 17:24:23 INFO mapred.JobClient:     Combine output records=3
08/07/04 17:24:23 INFO mapred.JobClient:     Reduce input groups=3
08/07/04 17:24:23 INFO mapred.JobClient:     Reduce input records=3
08/07/04 17:24:23 INFO mapred.JobClient:     Reduce output records=3
08/07/04 17:24:23 INFO mapred.JobClient:   File Systems
08/07/04 17:24:23 INFO mapred.JobClient:     Local bytes read=110873
08/07/04 17:24:23 INFO mapred.JobClient:     Local bytes written=83356
:::::Just after the job conf call:::::
Here I create the pdf file


And my cluster output, please check the bold part

CLUSTER
[rajesh@master hadoop-0.17.0]$ ./bin/hadoop jar hadoop-sdepoc-cluster1.jar
MyWordCount clsinput1 output2 june
08/07/04 17:48:25 INFO mapred.FileInputFormat: Total input paths to process
: 1
08/07/04 17:48:25 INFO mapred.JobClient: Running job: job_200807041747_0001
08/07/04 17:48:26 INFO mapred.JobClient:  map 0% reduce 0%
08/07/04 17:48:31 INFO mapred.JobClient:  map 50% reduce 0%
08/07/04 17:48:33 INFO mapred.JobClient:  map 100% reduce 0%
08/07/04 17:48:36 INFO mapred.JobClient:  map 100% reduce 100%
08/07/04 17:48:37 INFO mapred.JobClient: Job complete: job_200807041747_0001
08/07/04 17:48:37 INFO mapred.JobClient: Counters: 15
08/07/04 17:48:37 INFO mapred.JobClient:   Job Counters
08/07/04 17:48:37 INFO mapred.JobClient:     Launched map tasks=2
08/07/04 17:48:37 INFO mapred.JobClient:     Launched reduce tasks=1
08/07/04 17:48:37 INFO mapred.JobClient:     Data-local map tasks=2
08/07/04 17:48:37 INFO mapred.JobClient:   Map-Reduce Framework
08/07/04 17:48:37 INFO mapred.JobClient:     Map input records=316
08/07/04 17:48:37 INFO mapred.JobClient:     Map output records=0
08/07/04 17:48:37 INFO mapred.JobClient:     Map input bytes=27131
08/07/04 17:48:37 INFO mapred.JobClient:     Map output bytes=0
08/07/04 17:48:37 INFO mapred.JobClient:     Combine input records=0
08/07/04 17:48:37 INFO mapred.JobClient:     Combine output records=0
08/07/04 17:48:37 INFO mapred.JobClient:     Reduce input groups=0
08/07/04 17:48:37 INFO mapred.JobClient:     Reduce input records=0
08/07/04 17:48:37 INFO mapred.JobClient:     Reduce output records=0
08/07/04 17:48:37 INFO mapred.JobClient:   File Systems
08/07/04 17:48:37 INFO mapred.JobClient:     Local bytes read=117
08/07/04 17:48:37 INFO mapred.JobClient:     Local bytes written=367
08/07/04 17:48:37 INFO mapred.JobClient:     HDFS bytes read=29951
:::::Just after the job conf call:::::
[rajesh@master hadoop-0.17.0]$

Please suggest me some approach or inform if code needs to be uploaded.

-- 
View this message in context: http://www.nabble.com/Failed-to-get-reducer-output-while-running-in-cluster-tp18278864p18278864.html
Sent from the Hadoop core-user mailing list archive at Nabble.com.


Mime
View raw message