hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Zhijie Shen (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-3700) ATS Web Performance issue at load time when large number of jobs
Date Tue, 26 May 2015 19:23:23 GMT

    [ https://issues.apache.org/jira/browse/YARN-3700?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14559671#comment-14559671
] 

Zhijie Shen commented on YARN-3700:
-----------------------------------

[~xgong], thanks for updating the patch. The enhancement is exactly what I expect. Some comments
about the patch:

0. Can we change it to "max-applications" to "apps.num"?
{code}
String MAX_APP = "max-applications";
{code}

1. We can uniform getAllApplications and getAllApplications(long maxApps), as they are internal
interfaces. BTW, can we change maxApps to appsNum?
{code}
115	  @Override
116	  public Map<ApplicationId, ApplicationReport> getAllApplications(long maxApps)
117	      throws IOException {
118	    return getAllApplications();
{code}

2. AppBlock and AHSWebServices need to take in the param of appsNum. For AHSWebServices, we
may need to define a new param called "limit" to be as close as RMWebServices.

3. And we need to document the param and the new config in TimelineServer.md.

4. Would you mind rephrasing the description a bit?
{code}
1450	  /**
1451	   * Defines how many applications can be loaded into
1452	   * timeline service web ui.
1453	   * This will also affect the number of applications fetched from
1454	   * REST API and application history protocol.
1455	   */
{code}

5. I don't quite understand to following logic. Why do we want to use ID to set the timestamp?
And how do we verify if we limit the apps num, we are getting the latest apps.
{code}
477	    tEvent.setTimestamp(Integer.MAX_VALUE + 2L + appId.getId());
{code}

> ATS Web Performance issue at load time when large number of jobs
> ----------------------------------------------------------------
>
>                 Key: YARN-3700
>                 URL: https://issues.apache.org/jira/browse/YARN-3700
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>          Components: resourcemanager, webapp, yarn
>            Reporter: Xuan Gong
>            Assignee: Xuan Gong
>         Attachments: YARN-3700.1.patch, YARN-3700.2.1.patch, YARN-3700.2.2.patch, YARN-3700.2.patch
>
>
> Currently, we will load all the apps when we try to load the yarn timelineservice web
page. If we have large number of jobs, it will be very slow.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message