hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Inigo Goiri (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-3482) Report NM available resources in heartbeat
Date Thu, 16 Apr 2015 22:57:15 GMT

    [ https://issues.apache.org/jira/browse/YARN-3482?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14498926#comment-14498926
] 

Inigo Goiri commented on YARN-3482:
-----------------------------------

After some testing, I figured that it might be better to report the resources utilized by
the machine.
With this and the containers utilization, we can estimate the utilization of external processes.
In this way, there's no need for an external interface and the scheduler could take the right
decisions using the node utilization.
Thoughts?

> Report NM available resources in heartbeat
> ------------------------------------------
>
>                 Key: YARN-3482
>                 URL: https://issues.apache.org/jira/browse/YARN-3482
>             Project: Hadoop YARN
>          Issue Type: Improvement
>          Components: nodemanager, resourcemanager
>    Affects Versions: 2.7.0
>            Reporter: Inigo Goiri
>   Original Estimate: 504h
>  Remaining Estimate: 504h
>
> NMs are usually collocated with other processes like HDFS, Impala or HBase. To manage
this scenario correctly, YARN should be aware of the actual available resources. The proposal
is to have an interface to dynamically change the available resources and report this to the
RM in every heartbeat.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message