hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Inigo Goiri (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-3482) Report NM available resources in heartbeat
Date Fri, 17 Apr 2015 17:22:00 GMT

    [ https://issues.apache.org/jira/browse/YARN-3482?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14500223#comment-14500223
] 

Inigo Goiri commented on YARN-3482:
-----------------------------------

Makes sense. I think we should implement both and give the option to use one or the other.
Proposal for the names of the variables? yarn.nodemanager.track-utilization.node=true/false
yarn.nodemanager.track-utilization.containers=true/false
yarn.nodemanager.resource=true/false

(The second one would be for YARN-3481.)

For the interface, the simplest thing is to edit yarn.nodemanager.resource.cpu-vcores and
yarn.nodemanager.resource.memory-mb in yarn-site.xml. However, this implies modifying the
XML periodically which is kind of dirty for this purpose. I guess the cleanest is using the
admin interface, preferences?

> Report NM available resources in heartbeat
> ------------------------------------------
>
>                 Key: YARN-3482
>                 URL: https://issues.apache.org/jira/browse/YARN-3482
>             Project: Hadoop YARN
>          Issue Type: Improvement
>          Components: nodemanager, resourcemanager
>    Affects Versions: 2.7.0
>            Reporter: Inigo Goiri
>   Original Estimate: 504h
>  Remaining Estimate: 504h
>
> NMs are usually collocated with other processes like HDFS, Impala or HBase. To manage
this scenario correctly, YARN should be aware of the actual available resources. The proposal
is to have an interface to dynamically change the available resources and report this to the
RM in every heartbeat.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message