Mailing-List: contact core-user-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: core-user@hadoop.apache.org
Received-SPF: pass (athena.apache.org: domain of sd.codewarrior@gmail.com
 designates 209.85.217.170 as permitted sender)
DomainKey-Signature: a=rsa-sha1; c=nofws;
        d=gmail.com; s=gamma;
        h=mime-version:in-reply-to:references:date:message-id:subject:from:to
         :content-type;
        b=mWN1vvRTW+Pyn8u5O0NeFEWS63IHD4FUMva4QzkKAMvFnZmv4cSL2cVKLhEt40dCr5
         fDd/A8uuoJ4ibLKxsLb70MNGeqKy1W9BGC5onu5iFwnwgomFpziX5AcjHsiMaUfItuYc
         PisBsosJfaKl2PgFGaODfdfTkUKInXW+DCmtI=
MIME-Version: 1.0
In-Reply-To: <4af5cd780903131847x21f79eddxac96ec0653ae8828@mail.gmail.com>
References: <4af5cd780903131847x21f79eddxac96ec0653ae8828@mail.gmail.com>
Date: Fri, 13 Mar 2009 22:04:57 -0400
Message-ID: <4f10e2890903131904uf0d64k3ac7ccc44a21fe9e@mail.gmail.com>
Subject: Re: Controlling maximum # of tasks per node on per-job basis?
From: S D <sd.codewarrior@gmail.com>
To: core-user@hadoop.apache.org
Content-Type: multipart/alternative; boundary=0016e64eeb42e8bf3204650aa445

--0016e64eeb42e8bf3204650aa445
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: 7bit

I ran into this problem as well and several people on this list provided a
helpful response: once the tasktracker starts, the maximum number of tasks
per node can not be changed. In my case, I've solved this challenge by
stopping and starting mapred (stop-mapred.sh, start-mapred.sh) between jobs.
There is a jira so this may be changed in the future:  jira HADOOP-5170 (
http://issues.apache.org/jira/browse/HADOOP-5170)

John

On Fri, Mar 13, 2009 at 9:47 PM, Stuart White <stuart.white1@gmail.com>wrote:

> My cluster nodes have 2 dual-core processors, so, in general, I want
> to configure my nodes with a maximum of 3 task processes executed per
> node at a time.
>
> But, for some jobs, my tasks load large amounts of memory, and I
> cannot fit 3 such tasks on a single node.  For these jobs, I'd like to
> enforce running a maximum of 1 task process per node at a time.
>
> I've tried to enforce this by setting
> mapred.tasktracker.map.tasks.maximum at runtime, but I see it has no
> effect, because this is a configuration for the TaskTracker, which is
> of course already running before my job starts.
>
> Is there no way to configure a maximum # of map tasks per node on a
> per-job basis?
>
> Thanks!
>

--0016e64eeb42e8bf3204650aa445--