hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Varun Saxena (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-5845) Skip aclUpdated event publish to timelineserver or recovery
Date Mon, 07 Nov 2016 12:48:58 GMT

    [ https://issues.apache.org/jira/browse/YARN-5845?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15644090#comment-15644090
] 

Varun Saxena commented on YARN-5845:
------------------------------------

Thanks [~bibinchundatt]. Makes sense in not publishing an app ACLs' updated event on each
RM restart even for apps being recovered (especially finished apps). As you said, the timestamp
is different which would mean multiple entries in the store.
We republish the app created event on recovery too, to handle the case where app was created
and stored but the event was not sent to timeline server.
Basically, it seems as of now we send the App ACLs' updated event even before we send an app
created event which is a little weird.

I was thinking why not send app created event with ACLs' in application submission context.
Or if events have to be differentiated, send both the events together.
Thoughts ? 

> Skip aclUpdated event publish to timelineserver or recovery
> -----------------------------------------------------------
>
>                 Key: YARN-5845
>                 URL: https://issues.apache.org/jira/browse/YARN-5845
>             Project: Hadoop YARN
>          Issue Type: Bug
>            Reporter: Bibin A Chundatt
>            Assignee: Bibin A Chundatt
>            Priority: Critical
>
> Currently ACL update event is send to timeline server even on recovery {{RMAppManager#createAndPopulateNewRMApp}}.
> For 10K completed application when RM is restarted 10K ACL updated event is added to
timelinesever causing unnecessary over loading of system
> {code}
>     String appViewACLs = submissionContext.getAMContainerSpec()
>         .getApplicationACLs().get(ApplicationAccessType.VIEW_APP);
>     rmContext.getSystemMetricsPublisher().appACLsUpdated(
>         application, appViewACLs, System.currentTimeMillis());
> {code}
> *Events on each RM restart*
> {noformat}
>     "events": [{
>             "timestamp": 1478520292543,
>             "eventtype": "YARN_APPLICATION_ACLS_UPDATED",
>             "eventinfo": {}
>         }, {
>             "timestamp": 1478519600537,
>             "eventtype": "YARN_APPLICATION_ACLS_UPDATED",
>             "eventinfo": {}
>         }, {
>             "timestamp": 1478519557101,
>             "eventtype": "YARN_APPLICATION_ACLS_UPDATED",
>             "eventinfo": {}
>         }, 
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org


Mime
View raw message