hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Botong Huang (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-7599) [GPG] ApplicationCleaner in Global Policy Generator
Date Tue, 18 Sep 2018 17:46:00 GMT

    [ https://issues.apache.org/jira/browse/YARN-7599?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16619449#comment-16619449
] 

Botong Huang commented on YARN-7599:
------------------------------------

bq. One concern is the response size
Yeah we've hit this exact same problem in our federated cluster. That's why I added "RMWSConsts.DESELECTS"
(introduced in YARN-6280) when calling getApps to Router in _GPGUtils_. The _ResourceRequest_
entries occupy a significant portion of the appInfo response. In our prod cluster, this deselect
reduces the getApps call latency from 20 mins to seconds. 

> [GPG] ApplicationCleaner in Global Policy Generator
> ---------------------------------------------------
>
>                 Key: YARN-7599
>                 URL: https://issues.apache.org/jira/browse/YARN-7599
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>            Reporter: Botong Huang
>            Assignee: Botong Huang
>            Priority: Minor
>              Labels: federation, gpg
>         Attachments: YARN-7599-YARN-7402.v1.patch, YARN-7599-YARN-7402.v2.patch, YARN-7599-YARN-7402.v3.patch,
YARN-7599-YARN-7402.v4.patch
>
>
> In Federation, we need a cleanup service for StateStore as well as Yarn Registry. For
the former, we need to remove old application records. For the latter, failed and killed applications
might leave records in the Yarn Registry (see YARN-6128). We plan to do both cleanup work
in ApplicationCleaner in GPG



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org


Mime
View raw message