hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jianfeng (Jeff) Zhang" <jzh...@hortonworks.com>
Subject Re: trying to understand HashPartitioner
Date Wed, 18 Mar 2015 23:27:31 GMT

You can take it similar as the HashMap of java. Use the hashCode of one object to distribute
it into different bucket.

Best Regard,
Jeff Zhang

From: xeonmailinglist-gmail <xeonmailinglist@gmail.com<mailto:xeonmailinglist@gmail.com>>
Reply-To: "user@hadoop.apache.org<mailto:user@hadoop.apache.org>" <user@hadoop.apache.org<mailto:user@hadoop.apache.org>>
Date: Wednesday, March 18, 2015 at 7:08 PM
To: "user@hadoop.apache.org<mailto:user@hadoop.apache.org>" <user@hadoop.apache.org<mailto:user@hadoop.apache.org>>
Subject: Re: trying to understand HashPartitioner

What tells with partition will run on which reduce task?

On 18-03-2015 09:30, xeonmailinglist-gmail wrote:


I am trying to understand how HashPartitioner.java works. Thus, I ran a mapreduce job with
5 reducers and 5 input files. I thought that the output of getPartition(K2 key, V2 value,
int numReduceTasks) was the number of reduce task that K2 and V2 will execute. Is this correct?



View raw message