Return-Path: Delivered-To: apmail-hadoop-common-user-archive@www.apache.org Received: (qmail 88833 invoked from network); 31 Mar 2010 02:12:11 -0000 Received: from unknown (HELO mail.apache.org) (140.211.11.3) by 140.211.11.9 with SMTP; 31 Mar 2010 02:12:11 -0000 Received: (qmail 17494 invoked by uid 500); 31 Mar 2010 02:12:09 -0000 Delivered-To: apmail-hadoop-common-user-archive@hadoop.apache.org Received: (qmail 17337 invoked by uid 500); 31 Mar 2010 02:12:08 -0000 Mailing-List: contact common-user-help@hadoop.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: common-user@hadoop.apache.org Delivered-To: mailing list common-user@hadoop.apache.org Received: (qmail 17329 invoked by uid 99); 31 Mar 2010 02:12:08 -0000 Received: from nike.apache.org (HELO nike.apache.org) (192.87.106.230) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 31 Mar 2010 02:12:08 +0000 X-ASF-Spam-Status: No, hits=-0.7 required=10.0 tests=RCVD_IN_DNSWL_LOW,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (nike.apache.org: local policy) Received: from [65.55.88.11] (HELO TX2EHSOBE002.bigfish.com) (65.55.88.11) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 31 Mar 2010 02:11:59 +0000 Received: from mail151-tx2-R.bigfish.com (10.9.14.252) by TX2EHSOBE002.bigfish.com (10.9.40.22) with Microsoft SMTP Server id 8.1.240.5; Wed, 31 Mar 2010 02:11:37 +0000 Received: from mail151-tx2 (localhost.localdomain [127.0.0.1]) by mail151-tx2-R.bigfish.com (Postfix) with ESMTP id ADB885783F6 for ; Wed, 31 Mar 2010 02:11:37 +0000 (UTC) X-SpamScore: -18 X-BigFish: VPS-18(zz542N98dN9371Pzz1202hz31izz32i6bh2a8h43h61h) X-Spam-TCS-SCL: 0:0 Received: from mail151-tx2 (localhost.localdomain [127.0.0.1]) by mail151-tx2 (MessageSwitch) id 1270001496572933_25184; Wed, 31 Mar 2010 02:11:36 +0000 (UTC) Received: from TX2EHSMHS027.bigfish.com (unknown [10.9.14.252]) by mail151-tx2.bigfish.com (Postfix) with ESMTP id 7F0E81BF8051 for ; Wed, 31 Mar 2010 02:11:36 +0000 (UTC) Received: from ausb3extmailp01.amd.com (163.181.251.8) by TX2EHSMHS027.bigfish.com (10.9.99.127) with Microsoft SMTP Server (TLS) id 14.0.482.39; Wed, 31 Mar 2010 02:11:36 +0000 Received: from ausb3twp02.amd.com ([163.181.250.38]) by ausb3extmailp01.amd.com (Switch-3.2.7/Switch-3.2.7) with SMTP id o2V2AfAe009425 for ; Tue, 30 Mar 2010 21:10:44 -0500 X-WSS-ID: 0L04JF7-02-A1S-02 X-M-MSG: Received: from sausexhtp02.amd.com (sausexhtp02.amd.com [163.181.3.152]) (using TLSv1 with cipher RC4-MD5 (128/128 bits)) (No client certificate requested) by ausb3twp02.amd.com (Tumbleweed MailGate 3.7.2) with ESMTP id 21D38FCC243 for ; Tue, 30 Mar 2010 21:11:31 -0500 (CDT) Received: from SAUSEXMBP01.amd.com ([163.181.3.198]) by sausexhtp02.amd.com ([163.181.3.152]) with mapi; Tue, 30 Mar 2010 21:11:32 -0500 From: "Jones, Nick" To: "'common-user@hadoop.apache.org'" Date: Tue, 30 Mar 2010 21:11:32 -0500 Subject: Re: question on shuffle and sort Thread-Topic: question on shuffle and sort Thread-Index: AcrQdwUpQZDa8sT5R2azKMsFmsBgMAAAHYsd Message-ID: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: acceptlanguage: en-US Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-Reverse-DNS: unknown X-Virus-Checked: Checked by ClamAV on apache.org Something to keep in mind though, sorting is appropriate to the key type. T= ext will be sorted lexicographically. Nick Jones ----- Original Message ----- From: Ed Mazur To: common-user@hadoop.apache.org Sent: Tue Mar 30 21:07:29 2010=0A= Subject: Re: question on shuffle and sort On Tue, Mar 30, 2010 at 9:56 PM, Cui tony wrote: > =A0Did all key-value pairs of the map output, which have the same key, wi= ll > be sent to the same reducer tasknode? Yes, this is at the core of the MapReduce model. There is one call to the user reduce function per unique map output key. This grouping is achieved by sorting which means you see keys in increasing order. Ed