hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Eli Collins (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-1024) Define a virtual core unambigiously
Date Tue, 06 Aug 2013 17:02:49 GMT

    [ https://issues.apache.org/jira/browse/YARN-1024?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13730961#comment-13730961
] 

Eli Collins commented on YARN-1024:
-----------------------------------

bq. vcores are optional anyway (only used in DRF) 

Sandy corrected me offline that while this is true for the CS it is not true for the FS, which
by default (w/o DRF) will not schedule more containers worth of vcores than configured vcores
(which seems like it could lead to under-utilization given that the default resource calculator
only uses memory and not every container needs a whole core). By default the # vcores is the
# cores on the machine and MR asks containers w/ 1 vcore so we effectively have vcore=pcore
today as the default (re-inforced by the decision to remove the notion of pcore in YARN-782).
                
> Define a virtual core unambigiously
> -----------------------------------
>
>                 Key: YARN-1024
>                 URL: https://issues.apache.org/jira/browse/YARN-1024
>             Project: Hadoop YARN
>          Issue Type: Improvement
>            Reporter: Arun C Murthy
>            Assignee: Arun C Murthy
>
> We need to clearly define the meaning of a virtual core unambiguously so that it's easy
to migrate applications between clusters.
> For e.g. here is Amazon EC2 definition of ECU: http://aws.amazon.com/ec2/faqs/#What_is_an_EC2_Compute_Unit_and_why_did_you_introduce_it
> Essentially we need to clearly define a YARN Virtual Core (YVC).
> Equivalently, we can use ECU itself: *One EC2 Compute Unit provides the equivalent CPU
capacity of a 1.0-1.2 GHz 2007 Opteron or 2007 Xeon processor.*

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message