hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Botong Huang (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-7599) [GPG] ApplicationCleaner in Global Policy Generator
Date Tue, 18 Sep 2018 17:46:00 GMT

    [ https://issues.apache.org/jira/browse/YARN-7599?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16619449#comment-16619449

Botong Huang commented on YARN-7599:

bq. One concern is the response size
Yeah we've hit this exact same problem in our federated cluster. That's why I added "RMWSConsts.DESELECTS"
(introduced in YARN-6280) when calling getApps to Router in _GPGUtils_. The _ResourceRequest_
entries occupy a significant portion of the appInfo response. In our prod cluster, this deselect
reduces the getApps call latency from 20 mins to seconds. 

> [GPG] ApplicationCleaner in Global Policy Generator
> ---------------------------------------------------
>                 Key: YARN-7599
>                 URL: https://issues.apache.org/jira/browse/YARN-7599
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>            Reporter: Botong Huang
>            Assignee: Botong Huang
>            Priority: Minor
>              Labels: federation, gpg
>         Attachments: YARN-7599-YARN-7402.v1.patch, YARN-7599-YARN-7402.v2.patch, YARN-7599-YARN-7402.v3.patch,
> In Federation, we need a cleanup service for StateStore as well as Yarn Registry. For
the former, we need to remove old application records. For the latter, failed and killed applications
might leave records in the Yarn Registry (see YARN-6128). We plan to do both cleanup work
in ApplicationCleaner in GPG

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org

View raw message