hadoop-mapreduce-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From David Rosenstrauch <dar...@darose.net>
Subject Re: How to Influence Reduce Task Location.
Date Mon, 20 Dec 2010 03:28:10 GMT
And, as a follow-up, yes, we use the partitioner class to achieve this. 
  Our partioner runs a hashing algorithm which ensures that a given user 
key will always map to a specific shard #.

DR

On 12/18/2010 01:16 PM, Hari Sreekumar wrote:
> Hi Jane,
>
>           The partitioner class can be used to achieve this. (
> http://hadoop.apache.org/mapreduce/docs/r0.21.0/api/org/apache/hadoop/mapreduce/Partitioner.html
> ).
>
> Thanks,
> Hari
>
> On Sat, Dec 18, 2010 at 11:13 PM, Jane Chen<jxchen_us_1999@yahoo.com>wrote:
>
>> Hi All,
>>
>> Is there anyway to influence where a reduce task is run?  We have a case
>> where we'd like to choose the host to run the reduce task based on the
>> task's input key.
>>
>> Any suggestion is greatly appreciated.
>>
>> Thanks,
>> Jane

Mime
View raw message