hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jian He (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-4464) default value of yarn.resourcemanager.state-store.max-completed-applications should lower.
Date Thu, 19 May 2016 21:26:13 GMT

    [ https://issues.apache.org/jira/browse/YARN-4464?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15292175#comment-15292175
] 

Jian He commented on YARN-4464:
-------------------------------

The patch is changing the config for max number of completed apps RM keeps in memory, NOT
in store. Am I missing something ?

Also, is the intention to not keep any completed apps in store ?   IIRC, recovering 10k apps
from ZK/HDFS is taking about less than 30 seconds. [~magnum], are you sure the 20 min delay
is caused by recovering these 10k competed apps ?
bq. I think setting it to 0 might be better so users immediately realize they don't see any
completed apps after a restart.
[~kasha], why do you think it's better to not let users see the historical apps ? I feel it
is sometimes convenient to see the finished apps after restart. 

> default value of yarn.resourcemanager.state-store.max-completed-applications should lower.
> ------------------------------------------------------------------------------------------
>
>                 Key: YARN-4464
>                 URL: https://issues.apache.org/jira/browse/YARN-4464
>             Project: Hadoop YARN
>          Issue Type: Wish
>          Components: resourcemanager
>            Reporter: KWON BYUNGCHANG
>            Assignee: Daniel Templeton
>            Priority: Blocker
>         Attachments: YARN-4464.001.patch, YARN-4464.002.patch, YARN-4464.003.patch
>
>
> my cluster has 120 nodes.
> I configured RM Restart feature.
> {code}
> yarn.resourcemanager.recovery.enabled=true
> yarn.resourcemanager.store.class=org.apache.hadoop.yarn.server.resourcemanager.recovery.FileSystemRMStateStore
> yarn.resourcemanager.fs.state-store.uri=/system/yarn/rmstore
> {code}
> unfortunately I did not configure {{yarn.resourcemanager.state-store.max-completed-applications}}.
> so that property configured default value 10,000.
> I have restarted RM due to changing another configuartion.
> I expected that RM restart immediately.
> recovery process was very slow.  I have waited about 20min.  
> realize missing {{yarn.resourcemanager.state-store.max-completed-applications}}.
> its default value is very huge.  
> need to change lower value or document notice on [RM Restart page|http://hadoop.apache.org/docs/stable/hadoop-yarn/hadoop-yarn-site/ResourceManagerRestart.html].



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org


Mime
View raw message