hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From CTS-RAAJ <rajesh.moha...@cognizant.com>
Subject Re: NEED AN URGENT HELP PL, Failed to get reducer output while running in cluster
Date Sat, 05 Jul 2008 14:43:22 GMT


CTS-RAAJ wrote:
> 
> Hi All,
> 
> After all successful work aroud in getting the hadoop cluster run, I am
> kicked back by some problem while executig my program in cluster,
> Program takes input a log file, I have written a mapper which goes through
> the line to find certain pattern and pass the mapper output to reducer to
> sort and give me the counters. This I pass to a small job, which creates a
> chart using apache poi and itext api.
> 
> When I execute locally it runs fine with follwoing output
> 
> LOCAL
> 08/07/04 17:24:23 INFO mapred.LocalJobRunner:
> file:/home/rajesh/POC/hadoop-0.17.0/sdeinput1/SDE_LOG.log:0+27131
> 08/07/04 17:24:23 INFO mapred.TaskRunner: Task 'job_local_1_map_0000'
> done.
> 08/07/04 17:24:23 INFO mapred.TaskRunner: Saved output of task
> 'job_local_1_map_0000' to file:/home/rajesh/POC/hadoop-0.17.0/output1
> 08/07/04 17:24:23 INFO mapred.LocalJobRunner: reduce > reduce
> 08/07/04 17:24:23 INFO mapred.TaskRunner: Task 'reduce_75lmh0' done.
> 08/07/04 17:24:23 INFO mapred.TaskRunner: Saved output of task
> 'reduce_75lmh0' to file:/home/rajesh/POC/hadoop-0.17.0/output108/07/04
> 17:24:23 INFO mapred.JobClient: Job complete: job_local_1
> 08/07/04 17:24:23 INFO mapred.JobClient: Counters: 11
> 08/07/04 17:24:23 INFO mapred.JobClient:   Map-Reduce Framework
> 08/07/04 17:24:23 INFO mapred.JobClient:     Map input records=316
> 08/07/04 17:24:23 INFO mapred.JobClient:     Map output records=102
> 08/07/04 17:24:23 INFO mapred.JobClient:     Map input bytes=27131
> 08/07/04 17:24:23 INFO mapred.JobClient:     Map output bytes=816
> 08/07/04 17:24:23 INFO mapred.JobClient:     Combine input records=102
> 08/07/04 17:24:23 INFO mapred.JobClient:     Combine output records=3
> 08/07/04 17:24:23 INFO mapred.JobClient:     Reduce input groups=3
> 08/07/04 17:24:23 INFO mapred.JobClient:     Reduce input records=3
> 08/07/04 17:24:23 INFO mapred.JobClient:     Reduce output records=3
> 08/07/04 17:24:23 INFO mapred.JobClient:   File Systems
> 08/07/04 17:24:23 INFO mapred.JobClient:     Local bytes read=110873
> 08/07/04 17:24:23 INFO mapred.JobClient:     Local bytes written=83356
> :::::Just after the job conf call:::::
> Here I create the pdf file
> 
> 
> And my cluster output, please check the bold part
> 
> CLUSTER
> [rajesh@master hadoop-0.17.0]$ ./bin/hadoop jar hadoop-sdepoc-cluster1.jar
> MyWordCount clsinput1 output2 june
> 08/07/04 17:48:25 INFO mapred.FileInputFormat: Total input paths to
> process : 1
> 08/07/04 17:48:25 INFO mapred.JobClient: Running job:
> job_200807041747_0001
> 08/07/04 17:48:26 INFO mapred.JobClient:  map 0% reduce 0%
> 08/07/04 17:48:31 INFO mapred.JobClient:  map 50% reduce 0%
> 08/07/04 17:48:33 INFO mapred.JobClient:  map 100% reduce 0%
> 08/07/04 17:48:36 INFO mapred.JobClient:  map 100% reduce 100%
> 08/07/04 17:48:37 INFO mapred.JobClient: Job complete:
> job_200807041747_0001
> 08/07/04 17:48:37 INFO mapred.JobClient: Counters: 15
> 08/07/04 17:48:37 INFO mapred.JobClient:   Job Counters
> 08/07/04 17:48:37 INFO mapred.JobClient:     Launched map tasks=2
> 08/07/04 17:48:37 INFO mapred.JobClient:     Launched reduce tasks=1
> 08/07/04 17:48:37 INFO mapred.JobClient:     Data-local map tasks=2
> 08/07/04 17:48:37 INFO mapred.JobClient:   Map-Reduce Framework
> 08/07/04 17:48:37 INFO mapred.JobClient:     Map input records=316
> 08/07/04 17:48:37 INFO mapred.JobClient:     Map output records=0
> 08/07/04 17:48:37 INFO mapred.JobClient:     Map input bytes=27131
> 08/07/04 17:48:37 INFO mapred.JobClient:     Map output bytes=0
> 08/07/04 17:48:37 INFO mapred.JobClient:     Combine input records=0
> 08/07/04 17:48:37 INFO mapred.JobClient:     Combine output records=0
> 08/07/04 17:48:37 INFO mapred.JobClient:     Reduce input groups=0
> 08/07/04 17:48:37 INFO mapred.JobClient:     Reduce input records=0
> 08/07/04 17:48:37 INFO mapred.JobClient:     Reduce output records=0
> 08/07/04 17:48:37 INFO mapred.JobClient:   File Systems
> 08/07/04 17:48:37 INFO mapred.JobClient:     Local bytes read=117
> 08/07/04 17:48:37 INFO mapred.JobClient:     Local bytes written=367
> 08/07/04 17:48:37 INFO mapred.JobClient:     HDFS bytes read=29951
> :::::Just after the job conf call:::::
> [rajesh@master hadoop-0.17.0]$
> 
> Please suggest me some approach or inform if code needs to be uploaded.
> 
> 
:working::working::working::working:
-- 
View this message in context: http://www.nabble.com/Failed-to-get-reducer-output-while-running-in-cluster-tp18278864p18292904.html
Sent from the Hadoop core-user mailing list archive at Nabble.com.


Mime
View raw message