hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Aaron T. Myers (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (HDFS-2946) HA: Put a cap on the number of completed edits files retained by the NN
Date Wed, 03 Oct 2012 00:48:09 GMT

     [ https://issues.apache.org/jira/browse/HDFS-2946?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Aaron T. Myers updated HDFS-2946:
---------------------------------

    Attachment: HDFS-2946.patch

Here's a patch which addresses the issue. This adds a new configuration option called "dfs.namenode.max.extra.edits.segments.retained"
which will cap the number of edit log segments the NN retains, despite whatever is set as
the value of "dfs.namenode.num.extra.edits.retained".

This changed required the NNStorageRetentionManager be able to enumerate the edit log segments
available to it, so the bulk of the patch is plumbing that around.
                
> HA: Put a cap on the number of completed edits files retained by the NN
> -----------------------------------------------------------------------
>
>                 Key: HDFS-2946
>                 URL: https://issues.apache.org/jira/browse/HDFS-2946
>             Project: Hadoop HDFS
>          Issue Type: Improvement
>          Components: ha, name-node
>    Affects Versions: 2.0.1-alpha
>            Reporter: Aaron T. Myers
>            Assignee: Aaron T. Myers
>         Attachments: HDFS-2946.patch
>
>
> HDFS-2794 added a minimum number of transactions to retain in edits files. Since many
underlying file systems put a cap on the number of entries in a single directory, we should
put a cap on the number of edits files which will be retained by the NN.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message