hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hudson (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-5008) LeveldbRMStateStore database can grow substantially leading to long recovery times
Date Fri, 29 Apr 2016 04:35:12 GMT

    [ https://issues.apache.org/jira/browse/YARN-5008?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15263522#comment-15263522
] 

Hudson commented on YARN-5008:
------------------------------

FAILURE: Integrated in Hadoop-trunk-Commit #9689 (See [https://builds.apache.org/job/Hadoop-trunk-Commit/9689/])
YARN-5008. LeveldbRMStateStore database can grow substantially leading (jianhe: rev dd80042c42aadaa347db93028724f69c9aca69c6)
* hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/resources/yarn-default.xml
* hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/recovery/TestLeveldbRMStateStore.java
* hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/recovery/LeveldbRMStateStore.java
* hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/conf/YarnConfiguration.java


> LeveldbRMStateStore database can grow substantially leading to long recovery times
> ----------------------------------------------------------------------------------
>
>                 Key: YARN-5008
>                 URL: https://issues.apache.org/jira/browse/YARN-5008
>             Project: Hadoop YARN
>          Issue Type: Bug
>          Components: resourcemanager
>    Affects Versions: 2.7.0
>            Reporter: Jason Lowe
>            Assignee: Jason Lowe
>         Attachments: YARN-5008.001.patch
>
>
> On large clusters with high application churn the background compaction in leveldb may
not be able to keep up with the write rate.  This can lead to large leveldb databases that
take many minutes to recover despite not having very much real data in the database to load.
 Most the time is spent traversing tables full of keys that have been deleted.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org


Mime
View raw message