Return-Path: Delivered-To: apmail-hadoop-core-user-archive@www.apache.org Received: (qmail 11193 invoked from network); 19 May 2009 22:42:54 -0000 Received: from hermes.apache.org (HELO mail.apache.org) (140.211.11.3) by minotaur.apache.org with SMTP; 19 May 2009 22:42:54 -0000 Received: (qmail 24121 invoked by uid 500); 19 May 2009 19:53:07 -0000 Delivered-To: apmail-hadoop-core-user-archive@hadoop.apache.org Received: (qmail 24026 invoked by uid 500); 19 May 2009 19:53:07 -0000 Mailing-List: contact core-user-help@hadoop.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: core-user@hadoop.apache.org Delivered-To: mailing list core-user@hadoop.apache.org Received: (qmail 24016 invoked by uid 99); 19 May 2009 19:53:06 -0000 Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 19 May 2009 19:53:06 +0000 X-ASF-Spam-Status: No, hits=-0.0 required=10.0 tests=SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (athena.apache.org: domain of fossist@gmail.com designates 209.85.216.103 as permitted sender) Received: from [209.85.216.103] (HELO mail-px0-f103.google.com) (209.85.216.103) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 19 May 2009 19:52:58 +0000 Received: by pxi1 with SMTP id 1so11903pxi.5 for ; Tue, 19 May 2009 12:52:38 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:date:message-id:subject :from:to:content-type:content-transfer-encoding; bh=KWGIMs/7AFv2p1V9uduQptpYJyOCkrprhE6J/gagoxw=; b=EA8n0/82Exv5q8VxW0MJvci2vBBqTsADNOmoPP/sZ+82MKZp7W7nANHLrqhU2ie4dJ eJDFLynJ/EoxVV19RFAHrIjI0pZzbZJ9GWvOz4Bxud7RRlXrmHIye3l52HizhCT2UQcC wj7yBP2SwqHtWF26Id1CMF9sznLS2dp/7rK48= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:date:message-id:subject:from:to:content-type :content-transfer-encoding; b=Tsn57Z0C6h7jxwf9MyS9ea/dAUGHZ9ZoVrxF4tIYLEHedFAqD0CHdtvelxOaK80YJa 5CMoPkDjcfsLfAMDfpmEY5sRsDiHaxS8TxK+O7HBVK8ZuWHNs3HjmoLTI99R3rukCnIj 6zo3bfJSK9Drdp7Fbka3OT2WRH2pHMxC78Eoc= MIME-Version: 1.0 Received: by 10.143.1.12 with SMTP id d12mr133773wfi.203.1242762758229; Tue, 19 May 2009 12:52:38 -0700 (PDT) Date: Wed, 20 May 2009 01:22:38 +0530 Message-ID: <3f8297b20905191252t6a5d3602h74a614083018490f@mail.gmail.com> Subject: My configuration in conf/hadoop-site.xml is not being used. Why? From: Foss User To: "core-user@hadoop.apache.org" Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit X-Virus-Checked: Checked by ClamAV on apache.org I ran a job. In the jobtracker web interface, I found 4 maps and 1 reduce running. This is not what I set in my configuration files (hadoop-site.xml). My configuration file, conf/hadoop-site.xml is set as follows: mapred.map.tasks = 2 mapred.reduce.tasks = 2 However, the description of these properties mention that these settings would be ignored if mapred.job.tracker is set as 'local'. Mine is set properly with IP address, port number. Please note that the above configuration is from the 'conf/hadoop-site.xml' file of the job tracker node. I have also not overridden these settings in my Job class (java code). So, can anyone please explain why it was executing 4 maps but only 1 reduce? I have included some important entries from the job.xml of this job below: name value mapred.skip.reduce.max.skip.groups 0 mapred.reduce.max.attempts 4 mapred.reduce.tasks 1 mapred.reduce.tasks.speculative.execution true mapred.tasktracker.reduce.tasks.maximum 2 dfs.replication 2 mapred.reduce.copy.backoff 300 mapred.task.cache.levels 2 mapred.max.tracker.failures 4 mapred.map.tasks 4 mapred.map.tasks.speculative.execution true mapred.tasktracker.map.tasks.maximum 2 Please help.