Return-Path: X-Original-To: apmail-hadoop-hdfs-issues-archive@minotaur.apache.org Delivered-To: apmail-hadoop-hdfs-issues-archive@minotaur.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 6567FFD1B for ; Thu, 25 Apr 2013 13:08:37 +0000 (UTC) Received: (qmail 20065 invoked by uid 500); 25 Apr 2013 13:08:34 -0000 Delivered-To: apmail-hadoop-hdfs-issues-archive@hadoop.apache.org Received: (qmail 19993 invoked by uid 500); 25 Apr 2013 13:08:34 -0000 Mailing-List: contact hdfs-issues-help@hadoop.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: hdfs-issues@hadoop.apache.org Delivered-To: mailing list hdfs-issues@hadoop.apache.org Received: (qmail 18460 invoked by uid 99); 25 Apr 2013 13:08:28 -0000 Received: from arcas.apache.org (HELO arcas.apache.org) (140.211.11.28) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 25 Apr 2013 13:08:28 +0000 Date: Thu, 25 Apr 2013 13:08:28 +0000 (UTC) From: "Hudson (JIRA)" To: hdfs-issues@hadoop.apache.org Message-ID: In-Reply-To: References: Subject: [jira] [Commented] (HDFS-4739) NN can miscalculate the number of extra edit log segments to retain MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394 [ https://issues.apache.org/jira/browse/HDFS-4739?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13641754#comment-13641754 ] Hudson commented on HDFS-4739: ------------------------------ Integrated in Hadoop-Hdfs-trunk #1383 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk/1383/]) HDFS-4739. NN can miscalculate the number of extra edit log segments to retain. Contributed by Aaron T. Myers. (Revision 1471769) Result = FAILURE atm : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1471769 Files : * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NNStorageRetentionManager.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestNNStorageRetentionManager.java > NN can miscalculate the number of extra edit log segments to retain > ------------------------------------------------------------------- > > Key: HDFS-4739 > URL: https://issues.apache.org/jira/browse/HDFS-4739 > Project: Hadoop HDFS > Issue Type: Bug > Components: namenode > Affects Versions: 2.0.4-alpha > Reporter: Aaron T. Myers > Assignee: Aaron T. Myers > Fix For: 2.0.5-beta > > Attachments: HDFS-4739.patch, HDFS-4739.patch > > > The code in NNStorageRetentionManager#purgeOldStorage is intended to place a cap on the number of _extra_ edit log segments retained beyond what is strictly required to replay the FS history since the last fsimage. In fact this code currently places a limit on the _total_ number of extra edit log segments. If the number of required segments is greater than the configured cap, there will be no data loss, but an ugly error will be thrown and the NN will fail to start. > The fix is simple, and in the meantime a work-around is just to raise the value of dfs.namenode.max.extra.edits.segments.retained and start the NN. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira