hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Inigo Goiri (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-3482) Report NM available resources in heartbeat
Date Wed, 22 Apr 2015 01:24:59 GMT

    [ https://issues.apache.org/jira/browse/YARN-3482?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14506204#comment-14506204
] 

Inigo Goiri commented on YARN-3482:
-----------------------------------

I implemented an approximation to approach #2. The Node Manager has no admin interface so
I cannot leverage it to push the change of available resources. What I did was to extend the
NodeResourceMonitor to periodically read the yarn-site.xml and change the number of available
resources in the NM. This is then sent to the RM which is able to update those values.

Anybody has better ideas on what other interfaces I could use to push these changes to the
NM? The current approach works but it might be a little heavy as is periodically checking.

> Report NM available resources in heartbeat
> ------------------------------------------
>
>                 Key: YARN-3482
>                 URL: https://issues.apache.org/jira/browse/YARN-3482
>             Project: Hadoop YARN
>          Issue Type: Improvement
>          Components: nodemanager, resourcemanager
>    Affects Versions: 2.7.0
>            Reporter: Inigo Goiri
>   Original Estimate: 504h
>  Remaining Estimate: 504h
>
> NMs are usually collocated with other processes like HDFS, Impala or HBase. To manage
this scenario correctly, YARN should be aware of the actual available resources. The proposal
is to have an interface to dynamically change the available resources and report this to the
RM in every heartbeat.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message