Return-Path: 
 <core-user-return-5360-apmail-hadoop-core-user-archive=hadoop.apache.org@hadoop.apache.org>
Delivered-To: apmail-hadoop-core-user-archive@www.apache.org
Received: (qmail 4294 invoked from network); 20 Mar 2008 22:50:38 -0000
Received: from hermes.apache.org (HELO mail.apache.org) (140.211.11.2)
  by minotaur.apache.org with SMTP; 20 Mar 2008 22:50:38 -0000
Received: (qmail 54940 invoked by uid 500); 20 Mar 2008 22:50:32 -0000
Delivered-To: apmail-hadoop-core-user-archive@hadoop.apache.org
Received: (qmail 54915 invoked by uid 500); 20 Mar 2008 22:50:32 -0000
Mailing-List: contact core-user-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
List-Help: <mailto:core-user-help@hadoop.apache.org>
List-Unsubscribe: <mailto:core-user-unsubscribe@hadoop.apache.org>
List-Post: <mailto:core-user@hadoop.apache.org>
List-Id: <core-user.hadoop.apache.org>
Reply-To: core-user@hadoop.apache.org
Delivered-To: mailing list core-user@hadoop.apache.org
Received: (qmail 54906 invoked by uid 99); 20 Mar 2008 22:50:32 -0000
Received: from nike.apache.org (HELO nike.apache.org) (192.87.106.230)
    by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 20 Mar 2008 15:50:32 -0700
X-ASF-Spam-Status: No, hits=-0.0 required=10.0
	tests=SPF_PASS
X-Spam-Check-By: apache.org
Received-SPF: pass (nike.apache.org: domain of jimmy@indeed.com designates
 207.97.245.141 as permitted sender)
Received: from [207.97.245.141] (HELO smtp141.iad.emailsrvr.com)
 (207.97.245.141)
    by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 20 Mar 2008 22:49:42 +0000
Received: from relay4.r5.iad.mlsrvr.com (localhost [127.0.0.1])
	by relay4.r5.iad.mlsrvr.com (SMTP Server) with ESMTP id C9536C486
	for <core-user@hadoop.apache.org>; Thu, 20 Mar 2008 18:49:30 -0400 (EDT)
Received: by relay4.r5.iad.mlsrvr.com (Authenticated sender:
 jimmy-AT-indeed.com) with ESMTP id 98F01C385
	for <core-user@hadoop.apache.org>; Thu, 20 Mar 2008 18:49:30 -0400 (EDT)
Date: Thu, 20 Mar 2008 17:49:29 -0500
To: core-user@hadoop.apache.org
Subject: Re: Limiting Total # of TaskTracker threads
From: "Jimmy Wan" <jimmy@indeed.com>
Organization: Indeed.com
Content-Type: text/plain; format=flowed; delsp=yes; charset=iso-8859-15
MIME-Version: 1.0
References: <C405B200.3AB4C%tdunning@veoh.com>
Content-Transfer-Encoding: Quoted-Printable
Message-ID: <op.t8b60rwaj0ap09@jimmy.indeed.com>
In-Reply-To: <C405B200.3AB4C%tdunning@veoh.com>
User-Agent: Opera Mail/9.26 (Win32)
X-Virus-Checked: Checked by ClamAV on apache.org

On Tue, 18 Mar 2008 19:53:04 -0500, Ted Dunning <tdunning@veoh.com> wrot=
e:

> I think the original request was to limit the sum of maps and reduces =
 =

> rather than limiting the two parameters independently.

Ted, yes this is exactly what I'm looking for. I just found an issue tha=
t  =

seems to state that the old deprecated property is there, but it is not =
 =

documented:

https://issues.apache.org/jira/browse/HADOOP-2300

I tried using the max tasks in combination with setting the new values, =
 =

but that didn't seem to work. =3D( My machine labelled as "LIMITED MACHI=
NE"  =

had 2 maps and 1 reduce running at the same time.

The scenario I have is that I want to run multiple concurrent jobs throu=
gh  =

my cluster and have the CPU usage for that node be bound. Should I file =
a  =

new issue?

This was all with Hadoop 0.16.0

LIMITED MACHINE:
	<property>
	  <name>mapred.tasktracker.tasks.maximum</name>
	  <value>2</value>
	  <description>The maximum number of total tasks that will be run
	  simultaneously by a task tracker.
	  </description>
	</property>
	<property>
	  <name>mapred.tasktracker.map.tasks.maximum</name>
	  <value>1</value>
	  <description>The maximum number of map tasks that will be run
	  simultaneously by a task tracker.
	  </description>
	</property>
	<property>
	  <name>mapred.tasktracker.reduce.tasks.maximum</name>
	  <value>1</value>
	  <description>The maximum number of reduce tasks that will be run
	  simultaneously by a task tracker.
	  </description>
	</property>

OTHER CLUSTER MACHINES:
	<property>
	  <name>mapred.tasktracker.tasks.maximum</name>
	  <value>8</value>
	  <description>The maximum number of total tasks that will be run
	  simultaneously by a task tracker.
	  </description>
	</property>
	<property>
	  <name>mapred.tasktracker.map.tasks.maximum</name>
	  <value>4</value>
	  <description>The maximum number of map tasks that will be run
	  simultaneously by a task tracker.
	  </description>
	</property>
	<property>
	  <name>mapred.tasktracker.reduce.tasks.maximum</name>
	  <value>4</value>
	  <description>The maximum number of reduce tasks that will be run
	  simultaneously by a task tracker.
	  </description>
	</property>

> On 3/18/08 5:26 PM, "Arun C Murthy" <arunc@yahoo-inc.com> wrote:
>
>> The map/reduce tasks are not threads, they are run in separate JVMs
>> which are forked by the tasktracker.

Arun, yes, I did mean tasks, not threads.


-- =

Jimmy