hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Doug Cutting (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HADOOP-4295) job-level configurable mapred.map.tasks.maximum and mapred.reduce.tasks.maximum
Date Fri, 26 Sep 2008 23:15:44 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-4295?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12635062#action_12635062
] 

Doug Cutting commented on HADOOP-4295:
--------------------------------------

I'm not arguing that these are perfect, but permitting them to vary per node is a feature
that we shouldn't toss out.  Adding a different parameter that limits the number of tasks
that a job would actually run simultaneously on a node might be reasonable.  Thus I think
extending the scheduler, as is done in HADOOP-4035, is more like what we'd want here rather
than to change these existing parameters.

> job-level configurable mapred.map.tasks.maximum and mapred.reduce.tasks.maximum 
> --------------------------------------------------------------------------------
>
>                 Key: HADOOP-4295
>                 URL: https://issues.apache.org/jira/browse/HADOOP-4295
>             Project: Hadoop Core
>          Issue Type: Improvement
>          Components: mapred
>            Reporter: Christian Kunz
>
> Right now mapred.tasktracker.map.tasks.maximum and mapred.tasktracker.reduce.tasks.maximum
are set on the tasktracker level.
> In absense of a smart tasktracker monitoring resources and deciding in an adaptive manner
how many tasks can be run simultaneously, it would be nice to move these two configuration
options to the job level. This would make it easier to optimize the performance of a batch
of jobs.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message