Mailing-List: contact core-dev-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: core-dev@hadoop.apache.org
Message-ID: <904419063.1222660724399.JavaMail.jira@brutus>
Date: Sun, 28 Sep 2008 20:58:44 -0700 (PDT)
From: "Hemanth Yamijala (JIRA)" <jira@apache.org>
To: core-dev@hadoop.apache.org
Subject: [jira] Commented: (HADOOP-4295) job-level configurable
 mapred.map.tasks.maximum and mapred.reduce.tasks.maximum
In-Reply-To: <1087805662.1222459064263.JavaMail.jira@brutus>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit


    [ https://issues.apache.org/jira/browse/HADOOP-4295?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12635280#action_12635280 ] 

Hemanth Yamijala commented on HADOOP-4295:
------------------------------------------

bq. Should the title be changed to something like
bq. Modify the capacity scheduler (HADOOP-3445) to take job limitations concerning number of simultaneous tasks per node into account when scheduling tasks?

I am not sure if Doug was suggesting we use HADOOP-4035 to implement the functionality proposed in this JIRA. I understood it to mean that the approach should be the same. Either way, I think it would be nice to have it handled separately, since HADOOP-4035 is specifically addressing only memory based parameters in job control.

That said, I also think we'll need to consider unifying mechanisms of resource management at some time (maybe in the near future, *smile*). We already seem to have *slightly* different ways of dealing with cores, memory, and disk (a.k.a HADOOP-657) - specifying, measuring, reporting and scheduling.

> job-level configurable mapred.map.tasks.maximum and mapred.reduce.tasks.maximum 
> --------------------------------------------------------------------------------
>
>                 Key: HADOOP-4295
>                 URL: https://issues.apache.org/jira/browse/HADOOP-4295
>             Project: Hadoop Core
>          Issue Type: Improvement
>          Components: mapred
>            Reporter: Christian Kunz
>
> Right now mapred.tasktracker.map.tasks.maximum and mapred.tasktracker.reduce.tasks.maximum are set on the tasktracker level.
> In absense of a smart tasktracker monitoring resources and deciding in an adaptive manner how many tasks can be run simultaneously, it would be nice to move these two configuration options to the job level. This would make it easier to optimize the performance of a batch of jobs.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.