hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Arun C Murthy (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (YARN-2) Enhance CS to schedule accounting for both memory and cpu cores
Date Wed, 17 Oct 2012 07:48:06 GMT

     [ https://issues.apache.org/jira/browse/YARN-2?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Arun C Murthy updated YARN-2:
-----------------------------

    Attachment: YARN-2.patch

Thanks for the f/b Vinod & Bobby, I've incorporated most of your f/b:

# Renamed the interface and implmentations 
# Remove clusterResource from ResourceCalculator

I've kept the apis in Resources to ensure consistency of apis and to provide an indirection
to fix stuff later.

----

W.r.t #cores, I spent more time thinking about this - it seems to me we are better of leaving
the cores as integers and dynamically monitoring container usage and then over-subscribing
as necessary.

I don't see how users/applications can specify 1.8 cores or 1.38 cores etc. 

My thinking is that they can ask for integer cores and then the system (RM/NM) can monitoring
usage and dynamically over-subscribe. We can also discuss this separately in another jira.
Thoughts?
                
> Enhance CS to schedule accounting for both memory and cpu cores
> ---------------------------------------------------------------
>
>                 Key: YARN-2
>                 URL: https://issues.apache.org/jira/browse/YARN-2
>             Project: Hadoop YARN
>          Issue Type: New Feature
>          Components: capacityscheduler, scheduler
>            Reporter: Arun C Murthy
>            Assignee: Arun C Murthy
>             Fix For: 2.0.3-alpha
>
>         Attachments: MAPREDUCE-4327.patch, MAPREDUCE-4327.patch, MAPREDUCE-4327.patch,
MAPREDUCE-4327-v2.patch, MAPREDUCE-4327-v3.patch, MAPREDUCE-4327-v4.patch, MAPREDUCE-4327-v5.patch,
YARN-2-help.patch, YARN-2.patch, YARN-2.patch, YARN-2.patch
>
>
> With YARN being a general purpose system, it would be useful for several applications
(MPI et al) to specify not just memory but also CPU (cores) for their resource requirements.
Thus, it would be useful to the CapacityScheduler to account for both.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message