hadoop-mapreduce-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Arun C Murthy <...@yahoo-inc.com>
Subject Re: How to Influence Reduce Task Location.
Date Sun, 09 Jan 2011 09:11:08 GMT
You can't do this currently at all.

At a minimum you need a custom scheduler and you need to do more work  
on the JT for this to happen.

I still haven't seen a good enough reason for this 'feature' in nearly  
5 years.

Arun

On Dec 18, 2010, at 10:24 AM, Jane Chen wrote:

> But how does this help me request which host to schedule the reduce  
> task to?
> Thanks,
> Jane
>
> --- On Sat, 12/18/10, Hari Sreekumar <hsreekumar@clickable.com> wrote:
>
> From: Hari Sreekumar <hsreekumar@clickable.com>
> Subject: Re: How to Influence Reduce Task Location.
> To: mapreduce-user@hadoop.apache.org
> Date: Saturday, December 18, 2010, 10:16 AM
>
> Hi Jane,
>
>          The partitioner class can be used to achieve this. (http://hadoop.apache.org/mapreduce/docs/r0.21.0/api/org/apache/hadoop/mapreduce/Partitioner.html

> ).
>
> Thanks,
> Hari
>
> On Sat, Dec 18, 2010 at 11:13 PM, Jane Chen  
> <jxchen_us_1999@yahoo.com> wrote:
> Hi All,
>
> Is there anyway to influence where a reduce task is run?  We have a  
> case where we'd like to choose the host to run the reduce task based  
> on the task's input key.
>
> Any suggestion is greatly appreciated.
>
> Thanks,
> Jane
>
>
>
>
>


Mime
View raw message