hadoop-mapreduce-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jason Yang <lin.yang.ja...@gmail.com>
Subject Will all the intermediate output with the same key go to the same reducer?
Date Thu, 20 Sep 2012 13:12:08 GMT
Hi, all

I have a question that whether all the intermediate output with the same
key go to the same reducer or not?

If it is, in case of only two keys are generated from mapper, but there are
3 reducer running in this job, what would happen?

If not, how could I do some processing over the all data, like counting? I
think some would suggest to set the number of reducer to 1, but I thought
this would make the reducer to be the bottleneck when there are large
volume of intermediate output, isn't it?

-- 
YANG, Lin

Mime
View raw message