hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Rohith Sharma K S (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-5028) RMStateStore should trim down app state for completed applications
Date Fri, 16 Feb 2018 15:47:00 GMT

    [ https://issues.apache.org/jira/browse/YARN-5028?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16367486#comment-16367486
] 

Rohith Sharma K S commented on YARN-5028:
-----------------------------------------

Apologies for pitching in late! thanks [~grepas] for working on this JIRA. 

IIUC, the primary reason for this JIRA is reduce size of finished application data being stored
into RMStateStore. YARN-65 already reduces memory footprint for complected applications after
recovery. Most of the data resides on ApplicationSubmissionContext#AMContainerSpec. But current
patch is NOT removing AMContainerSpec from submission context. This brings me a doubt that
how much memory is being reduced storing from RMStateStore? Do you have any statical comparison
before and after the patch?

> RMStateStore should trim down app state for completed applications
> ------------------------------------------------------------------
>
>                 Key: YARN-5028
>                 URL: https://issues.apache.org/jira/browse/YARN-5028
>             Project: Hadoop YARN
>          Issue Type: Improvement
>          Components: resourcemanager
>    Affects Versions: 2.8.0
>            Reporter: Karthik Kambatla
>            Assignee: Gergo Repas
>            Priority: Major
>         Attachments: YARN-5028.000.patch, YARN-5028.001.patch, YARN-5028.002.patch, YARN-5028.003.patch,
YARN-5028.004.patch
>
>
> RMStateStore stores enough information to recover applications in case of a restart.
The store also retains this information for completed applications to serve their status to
REST, WebUI, Java and CLI clients. We don't need all the information we store today to serve
application status; for instance, we don't need the {{ApplicationSubmissionContext}}. 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org


Mime
View raw message