hadoop-mapreduce-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Mapred Learn <mapred.le...@gmail.com>
Subject question for understanding partitioning
Date Tue, 18 Jan 2011 20:09:34 GMT
hi,
I have a basic question. How does partitioning work ?

Following is a scenario I created to put up my question.

i) A parttition function is defined as partitioning map-output based on
aphabetical sorting of the key i.e. a partition for keys starting with 'a',
partition for keys starting with 'b'... partition for keys starting with
'z'. So, it means each map may have atmost 26 partitions ?

ii) What input will Reducer get ? Reducer will get first partition
(partition starting with 'a') of all the maps as it's input ? Does it mean
we will need 26 reduce tasks ?

Any inputs/documents/examples on this are appreciated. I am bit confused by
this.

Thanks in advance

Mime
View raw message