hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Vinod K V (JIRA)" <j...@apache.org>
Subject [jira] Commented: (MAPREDUCE-1018) Document changes to the memory management and scheduling model
Date Thu, 10 Jun 2010 06:10:16 GMT

    [ https://issues.apache.org/jira/browse/MAPREDUCE-1018?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12877336#action_12877336
] 

Vinod K V commented on MAPREDUCE-1018:
--------------------------------------

This patch needs to be split across projects after MAPREDUCE-1404. I tried doing it myself
for the sake of reviewing.

Few links are broken in capacity_scheduler.html and cluster_setup.html.

I couldn't merge changes to mapred_tutorial.xml easily, and so couldn't review it at all.

The changes in capacity_scheduler.xml and cluster_setup.xml mostly look good.

capacity_scheduler.xml
 - _"That is, the task is scheduled only if the following constraint is satisfied:"_
   Can we also somehow mention here that high memory jobs can occupy multiple slots? I felt
that isn't getting conveyed here, but was clear in Rahul's patch.
   If we do this, then we should also mention setup and cleanup tasks always occupying only
a single slot.

cluster_setup.xml
 - The documentation related to setting  java.opts and ulimits w.r.t memory usage is (accidentally?)
removed by your patch.

> Document changes to the memory management and scheduling model
> --------------------------------------------------------------
>
>                 Key: MAPREDUCE-1018
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-1018
>             Project: Hadoop Map/Reduce
>          Issue Type: Bug
>          Components: documentation
>    Affects Versions: 0.21.0
>            Reporter: Hemanth Yamijala
>            Assignee: Hemanth Yamijala
>            Priority: Blocker
>             Fix For: 0.21.0
>
>         Attachments: MAPRED-1018-1.patch, MAPRED-1018-2.patch, MAPRED-1018-3.patch, MAPRED-1018-4.patch.txt,
MAPRED-1018-5.patch.txt, MAPRED-1018-6.patch.txt, MAPRED-1018-7.patch.txt, MAPRED-1018-8.patch.txt,
MAPRED-1018-commons.patch
>
>
> There were changes done for the configuration, monitoring and scheduling of high ram
jobs. This must be documented in the mapred-defaults.xml and also on forrest documentation

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message