Mailing-List: contact user-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@hadoop.apache.org
Received-SPF: pass (athena.apache.org: domain of mogwaing@gmail.com designates
 209.85.220.176 as permitted sender)
MIME-Version: 1.0
In-Reply-To: 
 <CAOcnVr01P0UP=vKOa0nV1s1WY6J771iEs9eM6DeziDtrbVxVFQ@mail.gmail.com>
References: 
 <CAPDOW74WdbrJAj7mfOR12K0R3UtpO1CHPrWUyWKbj+J9cvXCng@mail.gmail.com>
	<CAOcnVr01P0UP=vKOa0nV1s1WY6J771iEs9eM6DeziDtrbVxVFQ@mail.gmail.com>
Date: Wed, 28 Nov 2012 15:09:49 +0900
Message-ID: 
 <CAPDOW75DVKf+pHZeT1Lw0xvHPHDNBorxkNb+eakk97AOSWscCg@mail.gmail.com>
Subject: Re: Assigning reduce tasks to specific nodes
From: Hiroyuki Yamada <mogwaing@gmail.com>
To: user@hadoop.apache.org
Content-Type: text/plain; charset=ISO-8859-1

Hi Harsh,

Thank you for the information.
I understand the current circumstances.

How about for mappers ?
As far as I tested, location information in InputSplit is ignored in 0.20.2,
so there seems no easy way for assigning mappers to specific nodes.
(I before checked the source and noticed that
location information is not restored when deserializing the InputSplit
instance.)

Thanks,
Hiroyuki

On Wed, Nov 28, 2012 at 2:08 PM, Harsh J <harsh@cloudera.com> wrote:
> This is not supported/available currently even in MR2, but take a look at
> https://issues.apache.org/jira/browse/MAPREDUCE-199.
>
>
> On Wed, Nov 28, 2012 at 9:34 AM, Hiroyuki Yamada <mogwaing@gmail.com> wrote:
>>
>> Hi,
>>
>> I am wondering how I can assign reduce tasks to specific nodes.
>> What I want to do is, for example,  assigning reducer which produces
>> part-00000 to node xxx000,
>> and part-00001 to node xxx001 and so on.
>>
>> I think it's abount task assignment scheduling but
>> I am not sure where to customize to achieve this.
>> Is this done by writing some extensions ?
>> or any easier way to do this ?
>>
>> Regards,
>> Hiroyuki
>
>
>
>
> --
> Harsh J