hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Naganarasimha G R (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-3616) determine how to generate YARN container events
Date Mon, 11 May 2015 18:03:00 GMT

    [ https://issues.apache.org/jira/browse/YARN-3616?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14538303#comment-14538303

Naganarasimha G R commented on YARN-3616:

I would like to continue working on this issue :).
Also to capture one important point from [~Vinodkv]'s review
bq. The missing dots occur when a container's life-cycle ends either on the RM or the AM.
We can take a dual pronged approach here? That or we make the RM-publisher itself a distributed
IMO dual pronged approach would be better, we can rely on NMs to post normal life cycle events
and in rare cases where NM cant handle, RM publish events directly to ATS.
And might be here distributed push might not work as in the cases which Vinod mentioned NM
might not be able to handle publishing as TimelineCollector might not be created as no container
is created in the NM side for that app. Correct me if i am wrong.

> determine how to generate YARN container events
> -----------------------------------------------
>                 Key: YARN-3616
>                 URL: https://issues.apache.org/jira/browse/YARN-3616
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>          Components: timelineserver
>    Affects Versions: YARN-2928
>            Reporter: Sangjin Lee
>            Assignee: Naganarasimha G R
> The initial design called for the node manager to write YARN container events to take
advantage of the distributed writes. RM acting as a sole writer of all YARN container events
would have significant scalability problems.
> Still, there are some types of events that are not captured by the NM. The current implementation
has both: RM writing container events and NM writing container events.
> We need to sort this out, and decide how we can write all needed container events in
a scalable manner.

This message was sent by Atlassian JIRA

View raw message