hadoop-mapreduce-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jianfeng (Jeff) Zhang" <jzh...@hortonworks.com>
Subject Re: trying to understand HashPartitioner
Date Wed, 18 Mar 2015 23:27:31 GMT

You can take it similar as the HashMap of java. Use the hashCode of one object to distribute
it into different bucket.



Best Regard,
Jeff Zhang


From: xeonmailinglist-gmail <xeonmailinglist@gmail.com<mailto:xeonmailinglist@gmail.com>>
Reply-To: "user@hadoop.apache.org<mailto:user@hadoop.apache.org>" <user@hadoop.apache.org<mailto:user@hadoop.apache.org>>
Date: Wednesday, March 18, 2015 at 7:08 PM
To: "user@hadoop.apache.org<mailto:user@hadoop.apache.org>" <user@hadoop.apache.org<mailto:user@hadoop.apache.org>>
Subject: Re: trying to understand HashPartitioner

What tells with partition will run on which reduce task?

On 18-03-2015 09:30, xeonmailinglist-gmail wrote:

Hi,

I am trying to understand how HashPartitioner.java works. Thus, I ran a mapreduce job with
5 reducers and 5 input files. I thought that the output of getPartition(K2 key, V2 value,
int numReduceTasks) was the number of reduce task that K2 and V2 will execute. Is this correct?

‚Äč

--
--


--
--
Mime
View raw message