hadoop-hdfs-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From JAY <jayunit...@gmail.com>
Subject Re: Assigning reduce tasks to specific nodes
Date Wed, 28 Nov 2012 15:31:29 GMT
Seems like hadoop is non optimal for this since it's designed to scale machines anonymously.
 

On Nov 27, 2012, at 11:08 PM, Harsh J <harsh@cloudera.com> wrote:

> This is not supported/available currently even in MR2, but take a look at https://issues.apache.org/jira/browse/MAPREDUCE-199.
> 
> 
> On Wed, Nov 28, 2012 at 9:34 AM, Hiroyuki Yamada <mogwaing@gmail.com> wrote:
> Hi,
> 
> I am wondering how I can assign reduce tasks to specific nodes.
> What I want to do is, for example,  assigning reducer which produces
> part-00000 to node xxx000,
> and part-00001 to node xxx001 and so on.
> 
> I think it's abount task assignment scheduling but
> I am not sure where to customize to achieve this.
> Is this done by writing some extensions ?
> or any easier way to do this ?
> 
> Regards,
> Hiroyuki
> 
> 
> 
> -- 
> Harsh J

Mime
View raw message