hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Naganarasimha G R (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-3045) [Event producers] Implement NM writing container lifecycle events to ATS
Date Thu, 30 Jul 2015 00:48:05 GMT

    [ https://issues.apache.org/jira/browse/YARN-3045?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14646992#comment-14646992
] 

Naganarasimha G R commented on YARN-3045:
-----------------------------------------

Thanks for the comments [~djp],
bq. We already have a new flush() API now for writer that checked in YARN-3949... You are
right that we are lacking of API to respect this priority/policy in the whole data flow for
writing. I will file another JIRA to track this.
I went through the discussions and the patch of YARN-3949, i feel calling two apis would be
not so user friendly and how will the users of TimelineClient call flush ? i think its not
captured in YARN-3949
bq. Anyway, I would support the scope (container events + foundation work) you proposed here
in case you are comfortable with.
I am fine with single jira, but only trouble is as and when the scope increases there will
be more delay in the jira as more discussions will be required(in this case which entity to
publish NM App & localization events ) and also as its been long since i am holding this
jira so thought of getting the basic one out and develop on top of it. I am ok if you want
to avoid multiple jira's.

bq. That's a good question. My initative thinking is we could need something like NodemanagerEntity
to store application events, resource localizaiton event, log aggregation handling events,
configuration, etc. However, I would like to hear you and other guys' ideas on this as well.
We had a discussion on this topic today in the meeting and [~sjlee0] was of the opinion not
to have another entity here. I think we need more discussions on this as it involves querying
too. Approach what i can think of is : 
* For Applicationlevel events in NM can be under ApplicationEntity and EventID can have Event
Type (INIT_APPLICATION/APPLICATION_FINISHED/APPLICATION_LOG_HANDLING_FAILED) and NM_ID 
* For Localization i feel it can be under ContainerEntity and the EventID can have  Event
Type (REQUEST,LOCALIZED,LOCALIZATION_FAILED)and PATH of the localized resource.

bq. IMO, the 2nd approach (hook to existing event dispatcher) looks simpler and straightforward.
This approach is straight fwd but not sure it might have impact( just initial apprehensions)
but will start of implementing for container events and share the initial patch based on this
approach.

> [Event producers] Implement NM writing container lifecycle events to ATS
> ------------------------------------------------------------------------
>
>                 Key: YARN-3045
>                 URL: https://issues.apache.org/jira/browse/YARN-3045
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>          Components: timelineserver
>            Reporter: Sangjin Lee
>            Assignee: Naganarasimha G R
>         Attachments: YARN-3045-YARN-2928.002.patch, YARN-3045-YARN-2928.003.patch, YARN-3045-YARN-2928.004.patch,
YARN-3045-YARN-2928.005.patch, YARN-3045-YARN-2928.006.patch, YARN-3045.20150420-1.patch
>
>
> Per design in YARN-2928, implement NM writing container lifecycle events and container
system metrics to ATS.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message