hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Inigo Goiri (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-3481) Report NM aggregated container resource utilization in heartbeat
Date Thu, 30 Apr 2015 06:06:06 GMT

    [ https://issues.apache.org/jira/browse/YARN-3481?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14520922#comment-14520922

Inigo Goiri commented on YARN-3481:

[~vinodkv], it looks like YARN-2965 is very similar to this. Actually, this also looks like
a clone to YARN-1012. Anyway, from what I understand, those JIRAs want to send utilization
metrics in the heartbeat and that's pretty much what I'm targeting here. My current prototype
extends ContainersMonitorImpl and puts this information into the NodeHealthStatus. I think
I could do that in any of those JIRAs. 

For now, I'm pushing the implementation of NodeResourceMonitor (YARN-3534) which will add
a ResourceUtilization entity so I'm not making progress here yet. Once I'm done with that
one, I can move to any of the other two JIRAs instead of keep moving forward with this one.

Please, let me know where you guys think it'd be better to push for this (even YARN-3332 is
a possibility).

> Report NM aggregated container resource utilization in heartbeat
> ----------------------------------------------------------------
>                 Key: YARN-3481
>                 URL: https://issues.apache.org/jira/browse/YARN-3481
>             Project: Hadoop YARN
>          Issue Type: Improvement
>          Components: nodemanager, resourcemanager
>    Affects Versions: 2.7.0
>            Reporter: Inigo Goiri
>   Original Estimate: 336h
>  Remaining Estimate: 336h
> To allow the RM take better scheduling decisions, it should be aware of the actual utilization
of the containers. The NM would aggregate the ContainerMetrics and report it in every heartbeat.
> Related to YARN-1012 but aggregated to reduce the heartbeat overhead.

This message was sent by Atlassian JIRA

View raw message