hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Naganarasimha G R (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-3044) [Event producers] Implement RM writing app lifecycle events to ATS
Date Wed, 08 Apr 2015 18:55:13 GMT

    [ https://issues.apache.org/jira/browse/YARN-3044?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14485797#comment-14485797

Naganarasimha G R commented on YARN-3044:

Thanks for the review comments [~zjshen],
bq. Can we use ContainerEntity. The events from RM are RM_XXXX_EVENT, and those from NM are
This approach should be fine, will update in the next patch. 

bq. I think we may overestimate the performance impact of writing NM lifecycles. Perhaps a
more reasonable performance metric is {{cost of writing lifecycle events per container / cost
of managing lifecycle per container * 100%}}. For example, if it is 2%, I guess it will probably
be acceptable.
Well true we might be underestimating the RM's ability to handle publishing of Container Entity.
But currently anyway have made it configurable to publish Container entities from RM side
and while measuring performance we can enable this and check the performance, if fine then
we can totally disable this configuration check and make RM publish always, your opinion ?

bq. I'm not sure if I understand this part correctly, but I incline that system timeline data
(RM/NM) is controlled by cluster config and per cluster, while application data is controlled
by framework or even per-application config. It may have some problem if the user is able
to change the former config. For example, he can hide its application information from cluster
may be i dint get this correctly, Is it that you intend to say that framework/cluster config
(which can impact the application execution) should be logged by RM/NM and other application
specific config can be logged by the AM ?

bq. Do you mean we should keep yarn.resourcemanager.system-metrics-publisher.enabled to control
RM SMP, and and create yarn.nodemanager.system-metrics-publisher.enabled to control NM SMP?
No i meant this commment of [~djp] {{We can have different entity types, e.g. NM_CONTAINER_EVENT,
RM_CONTAINER_EVENT, for containers' event get posted from NM or RM then we can fully understand
how the world could be different from NM and RM (i.e. start time, end time, etc.}} & {{However,
we can disable RM-side posting work in production environment by default.}}

> [Event producers] Implement RM writing app lifecycle events to ATS
> ------------------------------------------------------------------
>                 Key: YARN-3044
>                 URL: https://issues.apache.org/jira/browse/YARN-3044
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>          Components: timelineserver
>            Reporter: Sangjin Lee
>            Assignee: Naganarasimha G R
>         Attachments: YARN-3044.20150325-1.patch, YARN-3044.20150406-1.patch
> Per design in YARN-2928, implement RM writing app lifecycle events to ATS.

This message was sent by Atlassian JIRA

View raw message