From mapreduce-dev-return-3498-apmail-hadoop-mapreduce-dev-archive=hadoop.apache.org@hadoop.apache.org Thu Jun 2 14:39:41 2011 Return-Path: X-Original-To: apmail-hadoop-mapreduce-dev-archive@minotaur.apache.org Delivered-To: apmail-hadoop-mapreduce-dev-archive@minotaur.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 7248343E9 for ; Thu, 2 Jun 2011 14:39:41 +0000 (UTC) Received: (qmail 44701 invoked by uid 500); 2 Jun 2011 14:39:40 -0000 Delivered-To: apmail-hadoop-mapreduce-dev-archive@hadoop.apache.org Received: (qmail 44550 invoked by uid 500); 2 Jun 2011 14:39:39 -0000 Mailing-List: contact mapreduce-dev-help@hadoop.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: mapreduce-dev@hadoop.apache.org Delivered-To: mailing list mapreduce-dev@hadoop.apache.org Received: (qmail 44532 invoked by uid 99); 2 Jun 2011 14:39:39 -0000 Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 02 Jun 2011 14:39:39 +0000 X-ASF-Spam-Status: No, hits=3.3 required=5.0 tests=HTML_MESSAGE,NO_RDNS_DOTCOM_HELO,RCVD_IN_DNSWL_NONE,SPF_NEUTRAL X-Spam-Check-By: apache.org Received-SPF: neutral (athena.apache.org: local policy) Received: from [69.147.107.20] (HELO mrout1-b.corp.re1.yahoo.com) (69.147.107.20) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 02 Jun 2011 14:39:33 +0000 Received: from sp1-ex07cas03.ds.corp.yahoo.com (sp1-ex07cas03.ds.corp.yahoo.com [216.252.116.151]) by mrout1-b.corp.re1.yahoo.com (8.14.4/8.14.4/y.out) with ESMTP id p52EcsZU085370; Thu, 2 Jun 2011 07:38:54 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=yahoo-inc.com; s=cobra; t=1307025534; bh=gLJ6OdyEigsXLLz8OcKAE9OZUtyWVjvvDfZYkPgWnEY=; h=From:To:Date:Subject:Message-ID:In-Reply-To:Content-Type: MIME-Version; b=Jchl1ij/4mATMeE8HRz1V4KgtfJBPb+BR39gadT7ipAek7TAu1rKPc+ki2UUHAk4z bezyW8/XbZHtz8Mv0vb7OqZwkXg40b0flI+tx/eGmqcGqMPgeZXyvISW/gH5v8FSkY xBbLQT7ubIlZ79Rf6xyfz4DpSNmwNp4ZwyHd55uk= Received: from SP1-EX07VS02.ds.corp.yahoo.com ([216.252.116.135]) by sp1-ex07cas03.ds.corp.yahoo.com ([216.252.116.151]) with mapi; Thu, 2 Jun 2011 07:38:53 -0700 From: Robert Evans To: "mapreduce-user@hadoop.apache.org" , "raviteja@huawei.com" Date: Thu, 2 Jun 2011 07:38:51 -0700 Subject: Re: Reg ChainReducer usage Thread-Topic: Reg ChainReducer usage Thread-Index: Acwg2c3S65PQf4NTT3SlKjwf9GDAvwAMe9AgAAnDQ3I= Message-ID: In-Reply-To: Accept-Language: en-US Content-Language: en X-MS-Has-Attach: X-MS-TNEF-Correlator: acceptlanguage: en-US Content-Type: multipart/alternative; boundary="_000_CA0D0AAB24290evansyahooinccom_" MIME-Version: 1.0 --_000_CA0D0AAB24290evansyahooinccom_ Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable Moving to mapreduce user. Ravi, The issue is with the shuffle. The chain reducer cannot re-shuffle the out= put of a previous reducer. If you want that then you need to run a second = reduce only job. Instead usually the chain reducer would have a single red= ucer followed by 0 or more mappers, that can process the output of the redu= cer. --Bobby On 6/2/11 5:25 AM, "Ravi Teja" wrote: Hi, I Had some queries in the usage of the ChainReducer . 1)Only one reducer can be set. If we try to set the second reducer to the chain, IllegalArgumentException will be thrown. Then why is it a chainreducer ? 2)We have a option chain.reducer.byValue where in, it will decide whether the key value pair can be passed or not to the next Mapper/Reducer. But why is this property significant, as only reducer is called at last in the chain, no matter whatever the order is in the chain and there is nothin= g to pass to. Regards, Ravi Teja ***************************************************************************= * *********** This e-mail and attachments contain confidential information from HUAWEI, which is intended only for the person or entity whose address is listed above. Any use of the information contained herein in any way (including, but not limited to, total or partial disclosure, reproduction, or dissemination) by persons other than the intended recipient's) is prohibited. If you receive this e-mail in error, please notify the sender b= y phone or email immediately and delete it! --_000_CA0D0AAB24290evansyahooinccom_--