Mailing-List: contact hdfs-issues-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: hdfs-issues@hadoop.apache.org
Date: Wed, 3 Oct 2012 11:48:09 +1100 (NCT)
From: "Aaron T. Myers (JIRA)" <jira@apache.org>
To: hdfs-issues@hadoop.apache.org
Message-ID: <1800876133.157145.1349225289333.JavaMail.jiratomcat@arcas>
In-Reply-To: 
 <1372486429.34581.1329180299948.JavaMail.tomcat@hel.zones.apache.org>
Subject: [jira] [Updated] (HDFS-2946) HA: Put a cap on the number of
 completed edits files retained by the NN
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit


     [ https://issues.apache.org/jira/browse/HDFS-2946?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Aaron T. Myers updated HDFS-2946:
---------------------------------

    Attachment: HDFS-2946.patch

Here's a patch which addresses the issue. This adds a new configuration option called "dfs.namenode.max.extra.edits.segments.retained" which will cap the number of edit log segments the NN retains, despite whatever is set as the value of "dfs.namenode.num.extra.edits.retained".

This changed required the NNStorageRetentionManager be able to enumerate the edit log segments available to it, so the bulk of the patch is plumbing that around.
                
> HA: Put a cap on the number of completed edits files retained by the NN
> -----------------------------------------------------------------------
>
>                 Key: HDFS-2946
>                 URL: https://issues.apache.org/jira/browse/HDFS-2946
>             Project: Hadoop HDFS
>          Issue Type: Improvement
>          Components: ha, name-node
>    Affects Versions: 2.0.1-alpha
>            Reporter: Aaron T. Myers
>            Assignee: Aaron T. Myers
>         Attachments: HDFS-2946.patch
>
>
> HDFS-2794 added a minimum number of transactions to retain in edits files. Since many underlying file systems put a cap on the number of entries in a single directory, we should put a cap on the number of edits files which will be retained by the NN.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira