hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Junping Du (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-160) nodemanagers should obtain cpu/memory values from underlying OS
Date Mon, 18 Aug 2014 14:24:21 GMT

    [ https://issues.apache.org/jira/browse/YARN-160?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14100680#comment-14100680
] 

Junping Du commented on YARN-160:
---------------------------------

bq. The patch supports the old way.
Thanks for clarification here. Yes. I saw the details of getYARNContainerMemoryMB() which
sounds to honor previous NM resource configuration. 

bq. Isn't calculating the values from the hardware a better option?
Agree. But if the calculating results is not reasonable (like 0 or minus value), shall we
use previous NM default value instead? At least, experienced users (especially with test purpose)
already had some expectations even when they don't set any resource value here.

> nodemanagers should obtain cpu/memory values from underlying OS
> ---------------------------------------------------------------
>
>                 Key: YARN-160
>                 URL: https://issues.apache.org/jira/browse/YARN-160
>             Project: Hadoop YARN
>          Issue Type: Improvement
>          Components: nodemanager
>    Affects Versions: 2.0.3-alpha
>            Reporter: Alejandro Abdelnur
>            Assignee: Varun Vasudev
>             Fix For: 2.6.0
>
>         Attachments: apache-yarn-160.0.patch
>
>
> As mentioned in YARN-2
> *NM memory and CPU configs*
> Currently these values are coming from the config of the NM, we should be able to obtain
those values from the OS (ie, in the case of Linux from /proc/meminfo & /proc/cpuinfo).
As this is highly OS dependent we should have an interface that obtains this information.
In addition implementations of this interface should be able to specify a mem/cpu offset (amount
of mem/cpu not to be avail as YARN resource), this would allow to reserve mem/cpu for the
OS and other services outside of YARN containers.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

Mime
View raw message