hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Arun C Murthy (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-2) Enhance CS to schedule accounting for both memory and cpu cores
Date Mon, 24 Dec 2012 04:40:20 GMT

    [ https://issues.apache.org/jira/browse/YARN-2?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13539182#comment-13539182
] 

Arun C Murthy commented on YARN-2:
----------------------------------

Andrew - thanks for the review, I've incorporated most of your comments. I have a couple more
to go, but I wanted to put it up to discuss. In particular, thanks for catching SchedulerUtils.normalizeRequests.

Some responses:
* MultiResourceCalculator: I've chosen the current impl. to provide a slight perf advantage
where-in I don't need to compare 2nd-most dominant resource if I don't have to. Once we add
more resources it will be harder/uglier to do the same, but for now it seems worth it.
* I thought I already responded to Vinod, but I missed it - ratio and divide are actually
very different - one uses the notion of 'dominant resource' but the other doesn't (see MultiResourceCalculator)
and hence the need for two different apis.
                
> Enhance CS to schedule accounting for both memory and cpu cores
> ---------------------------------------------------------------
>
>                 Key: YARN-2
>                 URL: https://issues.apache.org/jira/browse/YARN-2
>             Project: Hadoop YARN
>          Issue Type: New Feature
>          Components: capacityscheduler, scheduler
>            Reporter: Arun C Murthy
>            Assignee: Arun C Murthy
>             Fix For: 2.0.3-alpha
>
>         Attachments: MAPREDUCE-4327.patch, MAPREDUCE-4327.patch, MAPREDUCE-4327.patch,
MAPREDUCE-4327-v2.patch, MAPREDUCE-4327-v3.patch, MAPREDUCE-4327-v4.patch, MAPREDUCE-4327-v5.patch,
YARN-2-help.patch, YARN-2.patch, YARN-2.patch, YARN-2.patch, YARN-2.patch
>
>
> With YARN being a general purpose system, it would be useful for several applications
(MPI et al) to specify not just memory but also CPU (cores) for their resource requirements.
Thus, it would be useful to the CapacityScheduler to account for both.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message