hadoop-yarn-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Rohith Sharma K S (JIRA)" <j...@apache.org>
Subject [jira] [Created] (YARN-4755) Optimize sending appACLsUpdated event to TimelineServer while recovering completed applications
Date Wed, 02 Mar 2016 10:03:18 GMT
Rohith Sharma K S created YARN-4755:
---------------------------------------

             Summary: Optimize sending appACLsUpdated event to TimelineServer while recovering
completed applications
                 Key: YARN-4755
                 URL: https://issues.apache.org/jira/browse/YARN-4755
             Project: Hadoop YARN
          Issue Type: Improvement
            Reporter: Rohith Sharma K S


In method {{RMAppManager#createAndPopulateNewRMApp}}, appACLsUpdated is sent to timelineserver
for every application that get created. 
{code}
 private RMAppImpl createAndPopulateNewRMApp(
      ApplicationSubmissionContext submissionContext, long submitTime,
      String user, boolean isRecovery) throws YarnException {
		//
                //
    String appViewACLs = submissionContext.getAMContainerSpec()
        .getApplicationACLs().get(ApplicationAccessType.VIEW_APP);
    rmContext.getSystemMetricsPublisher().appACLsUpdated(
        application, appViewACLs, System.currentTimeMillis());
    return application;
  }
{code}

Say if we have 10K completed applications to recover, 30K events will be generated i.e app_created,
app_finished and app_acl_updated. For completed applications, I think need not to send app-acl-updated
event with which gradually reduce load on the dispatcher. 

Eventhough MultiDispatcher is used to publish timeline events, it is bottle neck when max-completed
is configured very high value may be 100K.





--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message