hadoop-mapreduce-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Kamesh <kames...@imaginea.com>
Subject Re: Output of MAP Class only
Date Fri, 30 Sep 2011 12:25:06 GMT
On Friday 30 September 2011 04:17 PM, Rajen Bhatt (RBEI/EST1) wrote:
> Dear Experts:
> We are scaling down an image to few partitions and processing each 
> partition in the MAP class. We want to save the output of MAP class 
> only to demonstrate and present how this output sub images are looking 
> like. Can somebody give us a hint on how to only save the output of 
> MAP class (before we send them to the REDUCER class) ? We want to 
> demonstrate to the students the process of MAP REDUCE flow on simple 
> image processing operation. We are doing the attached images operation 
> using MAP REDUCE. We are sending the original image and getting as an 
> output the edge detected image. First one is the input Image, we are 
> splitting it, performing the edge detection on each split, and then 
> merging in the reducer.
> Thanks and Regards,
> ~~
> *Dr. Rajen Bhatt *
> *(*/*Corporate Research*/**/*@ Robert Bosch, India*/*)*
> Off: +91-80-4191-2025
> Mob: +91-9901241005

I have the following approach

At the reducer side, first write the map output to a file and then 
perform the reduce logic. After completion of the reduce logic again 
write the output to another file. Something like below

reduce(key,value, context) {
  mo.write(key,value,namedoutputfile1)
  reduce Logic
  mo.write(key,value after reduce logic, namedoutputfile2)
}

mo : an instance of Multiple Outputs

So here, namedoutputfile1 has mapper output and nameoutputfile2 has 
reducer output.

PS: The above approach works only when there is only one mapper from 
which reducer gets its input.


-- 
/Thanks&Regards,/
/Bh.V.S.Kamesh/

Mime
View raw message