Return-Path: X-Original-To: apmail-hadoop-common-user-archive@www.apache.org Delivered-To: apmail-hadoop-common-user-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id EDE5B1071B for ; Tue, 16 Jul 2013 07:05:36 +0000 (UTC) Received: (qmail 9176 invoked by uid 500); 16 Jul 2013 07:05:30 -0000 Delivered-To: apmail-hadoop-common-user-archive@hadoop.apache.org Received: (qmail 9026 invoked by uid 500); 16 Jul 2013 07:05:28 -0000 Mailing-List: contact user-help@hadoop.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@hadoop.apache.org Delivered-To: mailing list user@hadoop.apache.org Received: (qmail 8661 invoked by uid 99); 16 Jul 2013 07:05:26 -0000 Received: from nike.apache.org (HELO nike.apache.org) (192.87.106.230) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 16 Jul 2013 07:05:26 +0000 X-ASF-Spam-Status: No, hits=1.5 required=5.0 tests=HTML_MESSAGE,RCVD_IN_DNSWL_LOW,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (nike.apache.org: domain of devaraj.k@huawei.com designates 119.145.14.65 as permitted sender) Received: from [119.145.14.65] (HELO szxga02-in.huawei.com) (119.145.14.65) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 16 Jul 2013 07:05:19 +0000 Received: from 172.24.2.119 (EHLO szxeml210-edg.china.huawei.com) ([172.24.2.119]) by szxrg02-dlp.huawei.com (MOS 4.3.4-GA FastPath queued) with ESMTP id BEP62859; Tue, 16 Jul 2013 15:04:53 +0800 (CST) Received: from SZXEML420-HUB.china.huawei.com (10.82.67.159) by szxeml210-edg.china.huawei.com (172.24.2.183) with Microsoft SMTP Server (TLS) id 14.1.323.7; Tue, 16 Jul 2013 15:04:51 +0800 Received: from szxeml560-mbx.china.huawei.com ([169.254.3.63]) by szxeml420-hub.china.huawei.com ([10.82.67.159]) with mapi id 14.01.0323.007; Tue, 16 Jul 2013 15:04:51 +0800 From: Devaraj k To: "user@hadoop.apache.org" Subject: RE: Policies for placing a reducer Thread-Topic: Policies for placing a reducer Thread-Index: AQHOgdhl8PO3RS/hJUKpxM6ViRaJKplm4EOQ Date: Tue, 16 Jul 2013 07:04:49 +0000 Message-ID: <06006DDA5A27D541991944AC4117E7A96E1D8A1A@szxeml560-mbx.china.huawei.com> References: In-Reply-To: Accept-Language: en-US, zh-CN Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [10.18.168.85] Content-Type: multipart/alternative; boundary="_000_06006DDA5A27D541991944AC4117E7A96E1D8A1Aszxeml560mbxchi_" MIME-Version: 1.0 X-CFilter-Loop: Reflected X-Virus-Checked: Checked by ClamAV on apache.org --_000_06006DDA5A27D541991944AC4117E7A96E1D8A1Aszxeml560mbxchi_ Content-Type: text/plain; charset="gb2312" Content-Transfer-Encoding: base64 SGksDQoNCkl0IGRvZXNuoa90IGNvbnNpZGVyIHdoZXJlIHRoZSBtYXChr3MgcmFuIHRvIHNjaGVk dWxlIHRoZSByZWR1Y2VycyBiZWNhdXNlIHJlZHVjZXJzIG5lZWQgdG8gY29udGFjdCBhbGwgdGhl IG1hcHBlcnMgZm9yIHRoZSBtYXAgby9woa9zLiAgSXQgc2NoZWR1bGVzIHJlZHVjZXJzIHdoZXJl dmVyIHRoZSBzbG90cyBhdmFpbGFibGUuDQoNClRoYW5rcw0KRGV2YXJhaiBrDQoNCkZyb206IEZl bGl4LtDsIFttYWlsdG86eWduaHpldXNAZ21haWwuY29tXQ0KU2VudDogMTYgSnVseSAyMDEzIDA5 OjI1DQpUbzogdXNlckBoYWRvb3AuYXBhY2hlLm9yZw0KU3ViamVjdDogUG9saWNpZXMgZm9yIHBs YWNpbmcgYSByZWR1Y2VyDQoNCkhpIGFsbCwNCg0KV2hhdCBpcyB0aGUgcG9saWN5IG9mIGNob29z aW5nIGEgbm9kZSBmb3IgYSByZWR1Y2VyIGluIG1hcHJlZHVjZSAoSGFkb29wIHYxLjIuMCk/DQpG b3IgZXhhbXBsZSwNCklmIGEgY2x1c3RlciBoYXMgNSBzbGF2ZXMsIGVhY2ggc2xhdmUgY2FuIHNl cnZlIDIgbWFwcyBhbmQgMiByZWR1Y2VzICwgdGhlcmUgaXMgYSBqb2Igd2hvIG9jY3VwaWVzIDUg bWFwcGVycyBhbmQgMyByZWR1Y2VycyAsIGhvdyBkb2VzIHRoZSBqb2J0cmFja2VyIGFzc2lnbiBy ZWR1Y2VycyB0byB0aGVzZSBub2RlcyAoY2hvb3NpbmcgZnJlZSBzbGF2ZXMgb3IgcGxhY2luZyBy ZWR1Y2VycyBjbG9zZSB0byBtYXBwZXJzKT8NCg0KVGhhbmtzLg0K --_000_06006DDA5A27D541991944AC4117E7A96E1D8A1Aszxeml560mbxchi_ Content-Type: text/html; charset="gb2312" Content-Transfer-Encoding: quoted-printable

Hi,

 <= /p>

It doesn=A1=AFt consider where the map=A1=AFs ran to schedule the reduce= rs because reducers need to contact all the mappers for the map o/p=A1=AFs.  It schedules reducers wherever the slots available.

 

Thanks<= /p>

Devaraj k

 <= /p>

From: Felix.=D0=EC [mailto:ygnhzeus@gmail.com]
Sent: 16 July 2013 09:25
To: user@hadoop.apache.org
Subject: Policies for placing a reducer

 

Hi all,

 

What is the policy of choosing a node for a reducer = in mapreduce (Hadoop v1.2.0)? 

For example,

If a cluster has 5 slaves, each slave can serve 2 ma= ps and 2 reduces , there is a job who occupies 5 mappers and 3 reducers , h= ow does the jobtracker assign reducers to these nodes (choosing free slaves= or placing reducers close to mappers)? 

 

Thanks.

--_000_06006DDA5A27D541991944AC4117E7A96E1D8A1Aszxeml560mbxchi_--