hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jian He (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-1378) Implement a RMStateStore cleaner for deleting application/attempt info
Date Wed, 13 Nov 2013 21:01:23 GMT

    [ https://issues.apache.org/jira/browse/YARN-1378?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13821828#comment-13821828
] 

Jian He commented on YARN-1378:
-------------------------------

Did the experiment with ZK in my local machine, storing 5000 application state data takes
about 13- 14 secs, and loading 5000 applications state data from ZK takes 2-3 secs. This does
not include any attempt data.
We can introduce one more configuration for limiting the number of completed applications
stored in disk if limited.

> Implement a RMStateStore cleaner for deleting application/attempt info
> ----------------------------------------------------------------------
>
>                 Key: YARN-1378
>                 URL: https://issues.apache.org/jira/browse/YARN-1378
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>          Components: resourcemanager
>            Reporter: Jian He
>            Assignee: Jian He
>         Attachments: YARN-1378.1.patch, YARN-1378.2.patch, YARN-1378.3.patch
>
>
> Now that we are storing the final state of application/attempt instead of removing application/attempt
info on application/attempt completion(YARN-891), we need a separate RMStateStore cleaner
for cleaning the application/attempt state.



--
This message was sent by Atlassian JIRA
(v6.1#6144)

Mime
View raw message