hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Vrushali C (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-6121) Launch app collectors for unmanaged AMs'
Date Thu, 07 Jun 2018 18:00:01 GMT

    [ https://issues.apache.org/jira/browse/YARN-6121?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16505015#comment-16505015

Vrushali C commented on YARN-6121:

We discussed this briefly in the community call today between [~abmodi] [~Sushil-K-S] Prabha
[~haibochen] [~rohithsharma] and I. 

For the federation use case, it is good for the containers in other clusters to connect with
the collector related to the original managed AM rather than connecting with other collectors
related to the unmanaged AMs. The reason is that app level aggregations are done in memory
in the timeline collector per application and then written to the backend. If this aggregation
gets split between collectors for the same app id, then the counts will not be correct per

Regardless of how we solve the unmanaged AM's collector, federation should continue to use
the existing way of piggy backing on the AMRM Proxy to help the containers on other clusters
talk to the collector on original cluster.

> Launch app collectors for unmanaged AMs'
> ----------------------------------------
>                 Key: YARN-6121
>                 URL: https://issues.apache.org/jira/browse/YARN-6121
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>            Reporter: Varun Saxena
>            Assignee: Abhishek Modi
>            Priority: Major
> Currently an app collector is launched whenever an AM container is launched on a NM.
This means for an unmanaged AM, app collector is never launched.

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org

View raw message