hadoop-mapreduce-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From David Rosenstrauch <dar...@darose.net>
Subject Re: question for understanding partitioning
Date Tue, 18 Jan 2011 20:32:33 GMT
On 01/18/2011 03:09 PM, Mapred Learn wrote:
> hi,
> I have a basic question. How does partitioning work ?
> Following is a scenario I created to put up my question.
> i) A parttition function is defined as partitioning map-output based on
> aphabetical sorting of the key i.e. a partition for keys starting with 'a',
> partition for keys starting with 'b'... partition for keys starting with
> 'z'. So, it means each map may have atmost 26 partitions ?
> ii) What input will Reducer get ? Reducer will get first partition
> (partition starting with 'a') of all the maps as it's input ? Does it mean
> we will need 26 reduce tasks ?
> Any inputs/documents/examples on this are appreciated. I am bit confused by
> this.
> Thanks in advance

You should probably read the Yahoo tutorial to brush up on the topic 
before asking on the list.


If you still don't understand after that, and you post a specific 
question (i.e., not "how does partitioning work") I'm sure someone will 
be able to answer.


View raw message