Return-Path: Delivered-To: apmail-hadoop-common-user-archive@www.apache.org Received: (qmail 14635 invoked from network); 30 Jun 2010 09:35:48 -0000 Received: from unknown (HELO mail.apache.org) (140.211.11.3) by 140.211.11.9 with SMTP; 30 Jun 2010 09:35:48 -0000 Received: (qmail 77227 invoked by uid 500); 30 Jun 2010 09:35:46 -0000 Delivered-To: apmail-hadoop-common-user-archive@hadoop.apache.org Received: (qmail 76736 invoked by uid 500); 30 Jun 2010 09:35:42 -0000 Mailing-List: contact common-user-help@hadoop.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: common-user@hadoop.apache.org Delivered-To: mailing list common-user@hadoop.apache.org Received: (qmail 76728 invoked by uid 99); 30 Jun 2010 09:35:41 -0000 Received: from nike.apache.org (HELO nike.apache.org) (192.87.106.230) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 30 Jun 2010 09:35:41 +0000 X-ASF-Spam-Status: No, hits=2.2 required=10.0 tests=FREEMAIL_FROM,HTML_MESSAGE,SPF_PASS,T_TO_NO_BRKTS_FREEMAIL X-Spam-Check-By: apache.org Received-SPF: pass (nike.apache.org: domain of pierreact@gmail.com designates 74.125.82.176 as permitted sender) Received: from [74.125.82.176] (HELO mail-wy0-f176.google.com) (74.125.82.176) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 30 Jun 2010 09:35:34 +0000 Received: by wyb29 with SMTP id 29so836580wyb.35 for ; Wed, 30 Jun 2010 02:35:14 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:received:date:message-id :subject:from:to:content-type; bh=/BjZX0IImeCipQy4dn4zWgv0prwSu7gfYRaZhn6G+XU=; b=UYI/b7SMZnNfbjuxfXDTaeIy7zrZO4fhJxtBnZe6P+uyvd6RGwvaURbmhomj7r4Txo K3Nhg3nERfCW5uGP2d0cEMtPj/Nhoiqs5wntrJ/d5PRQb2qO3YOOFpsWlXaVZtrRCTBN L11Em3NRekC4lXN5dhpKLAli+uqa/EtpG2gaY= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:date:message-id:subject:from:to:content-type; b=pfAKowf1cp3sneLsTr3hVL+Hg0st/N0x9TSwRx7aQ/z8uHA2Fwh55+L6ryfCL3HVfl ywUWHIXgwNXg2xzgK2+9y2GQkUn8zhqd5sEdMfXW3hu5nDKCDJ2Ru2gTNSxt3PkqSrZe W96mus3EhWrRXq0/11HIzVh8jSndr5pQykM2g= MIME-Version: 1.0 Received: by 10.216.90.13 with SMTP id d13mr6864009wef.18.1277890514369; Wed, 30 Jun 2010 02:35:14 -0700 (PDT) Received: by 10.216.154.144 with HTTP; Wed, 30 Jun 2010 02:35:14 -0700 (PDT) Date: Wed, 30 Jun 2010 11:35:14 +0200 Message-ID: Subject: Dynamically set mapred.tasktracker.map.tasks.maximum from inside a job. From: Pierre ANCELOT To: common-user@hadoop.apache.org Content-Type: multipart/alternative; boundary=0016e6d7dff52bc3a6048a3c12f6 X-Virus-Checked: Checked by ClamAV on apache.org --0016e6d7dff52bc3a6048a3c12f6 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Hi everyone :) There's something I'm probably doing wrong but I can't seem to figure out what. I have two hadoop programs running one after the other. This is done because they don't have the same needs in term of processor in memory, so by separating them I optimize each task better. Fact is, I need for the first job on every node mapred.tasktracker.map.tasks.maximum set to 12. For the second task, I need it to be set to 20. so by default I set it to 12 and in the second job's code, I set this: Configuration hadoopConfiguration =3D new Configuration(); hadoopConfiguration.setInt("mapred.tasktracker.map.tasks.maximum", 20); But when running the job, instead of having the 20 tasks on each node as expected, I have 12.... Any idea please? Thank you. Pierre. --=20 http://www.neko-consulting.com Ego sum quis ego servo "Je suis ce que je prot=E8ge" "I am what I protect" --0016e6d7dff52bc3a6048a3c12f6--