hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Vinod K V (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HADOOP-5881) Simplify configuration related to task-memory-monitoring and memory-based scheduling
Date Tue, 26 May 2009 09:02:45 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-5881?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12712919#action_12712919
] 

Vinod K V commented on HADOOP-5881:
-----------------------------------

Here's the proposal:
|| *Parameter* || *Type* || *Meaning* ||
| mapred.cluster.map.memory.mb  | set by admin, cluster-wide | Cluster definition of memory
per map slot. The maximum amount of memory, in MB, each map task on a tasktracker can consume.|
| mapred.cluster.reduce.memory.mb |set by admin, cluster-wide | Cluster definition of memory
per reduce slot. The maximum amount of memory, in MB, each reduce task on a tasktracker can
consume. |
| mapred.job.map.memory.mb | set by user, per-job | Job requirement for map tasks. The maximum
amount of memory each map task of a job can consume, in MB. | 
| mapred.job.reduce.memory.mb | set by user, per-job | job requirement for reduce tasks. The
maximum amount of memory each reduce task of a job can consume, in MB. |
|mapred.cluster.max.map.memory.mb | set by admin, cluster-wide | Max limit on jobs. The maximum
value that can be specified by a user via mapred.job.map.memory.mb, in MB. A job that asks
for more than this number will be failed at submission itself.|
|mapred.cluster.max.reduce.memory.mb | set by admin, cluster-wide | Max limit on jobs. The
maximum value that can be specified by a user via mapred.job.reduce.memory.mb, in MB. A job
that asks for more than this number will be failed at submission itself.|

> Simplify configuration related to task-memory-monitoring and memory-based scheduling
> ------------------------------------------------------------------------------------
>
>                 Key: HADOOP-5881
>                 URL: https://issues.apache.org/jira/browse/HADOOP-5881
>             Project: Hadoop Core
>          Issue Type: Bug
>          Components: mapred
>            Reporter: Vinod K V
>
> The configuration we have now is pretty complicated. Besides everything else, the mechanism
of not specifying parameters separately for maps and reduces leads to problems like HADOOP-5811.
This JIRA should address simplifying things and at the same time solving these problems.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message