Return-Path: X-Original-To: apmail-hadoop-mapreduce-user-archive@minotaur.apache.org Delivered-To: apmail-hadoop-mapreduce-user-archive@minotaur.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id C983A9AA6 for ; Fri, 6 Apr 2012 21:33:10 +0000 (UTC) Received: (qmail 82935 invoked by uid 500); 6 Apr 2012 21:33:09 -0000 Delivered-To: apmail-hadoop-mapreduce-user-archive@hadoop.apache.org Received: (qmail 82878 invoked by uid 500); 6 Apr 2012 21:33:09 -0000 Mailing-List: contact mapreduce-user-help@hadoop.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: mapreduce-user@hadoop.apache.org Delivered-To: mailing list mapreduce-user@hadoop.apache.org Received: (qmail 82870 invoked by uid 99); 6 Apr 2012 21:33:09 -0000 Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 06 Apr 2012 21:33:09 +0000 X-ASF-Spam-Status: No, hits=1.5 required=5.0 tests=HTML_MESSAGE,RCVD_IN_DNSWL_LOW,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (athena.apache.org: domain of piyush.kansal@gmail.com designates 74.125.82.48 as permitted sender) Received: from [74.125.82.48] (HELO mail-wg0-f48.google.com) (74.125.82.48) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 06 Apr 2012 21:33:03 +0000 Received: by wgbdq10 with SMTP id dq10so2323503wgb.29 for ; Fri, 06 Apr 2012 14:32:42 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:date:message-id:subject:from:to:content-type; bh=nTONtLWrAg/Bts3JS0qDDXNGON4LIdcQFkXhoOaH0DM=; b=IqyE2C6cMrfpcDEvSEckB4o3WBv0zM9i5SjG+DGPU4K/dDX5I2hUY/6NZFxO3IhPLM zbuc1GE2q67RxGeKf2ibzCYa6yhRqOxZSWh3p/d5aF4prZDUSTBSMknwWppzS1aYhs0a 19xjHpihovdmeijgXmIqSOnbLTzCpZHI+Ee7GvwFUOaneFPCoNVCpTlI1hQpvc5twB3I IilW/NGnBV2o7LYq3xAHdwkf73aYMF2Crl7JUz7zjjkeqMIa/2A09WazNtlqTR1vY+lB vByGR/hOtKr/Vmgy71HGwu1qaUK5knnuom/VmJ4/qrqlNzy4AZnZz5gHk64sttfX5KNs eRmA== MIME-Version: 1.0 Received: by 10.180.95.129 with SMTP id dk1mr16146430wib.3.1333747962009; Fri, 06 Apr 2012 14:32:42 -0700 (PDT) Received: by 10.216.203.32 with HTTP; Fri, 6 Apr 2012 14:32:41 -0700 (PDT) Date: Fri, 6 Apr 2012 17:32:41 -0400 Message-ID: Subject: v0.20.203: Do we need to set the number of reducers everytime? From: Piyush Kansal To: mapreduce-user@hadoop.apache.org Content-Type: multipart/alternative; boundary=f46d0444ee2d7ed68604bd09650f X-Virus-Checked: Checked by ClamAV on apache.org --f46d0444ee2d7ed68604bd09650f Content-Type: text/plain; charset=ISO-8859-1 Hi, I am using Partitioner and Grouper class in my program. And lets say the data which I want to process using MapReduce varies in size, it can be just 10MB or can go up to 10GB. So, do I need to set the number of reducers depending on data size, every time I want to run MapReduce. Or the Hadoop framework automatically invokes the required number of reducers at run time? -- Regards, Piyush Kansal --f46d0444ee2d7ed68604bd09650f Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Hi,

I am using Partitioner and Grouper class in my program. And lets= say the data which I want to process using MapReduce varies in size, it ca= n be just 10MB or can go up to 10GB.
So, do I need to set the number of = reducers depending on data size, every time I want to run MapReduce. Or the= Hadoop framework automatically invokes the required number of reducers at = run time?

--
Regards,
Piyush Kansal

--f46d0444ee2d7ed68604bd09650f--