hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Eli Collins (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-2946) HA: Put a cap on the number of completed edits files retained by the NN
Date Wed, 10 Oct 2012 20:29:03 GMT

    [ https://issues.apache.org/jira/browse/HDFS-2946?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13473532#comment-13473532
] 

Eli Collins commented on HDFS-2946:
-----------------------------------

Nice change ATM.  +1 modulo two nits:

- purgeOldStorage could use a quick comment something like "First determine all the extra
edits to retain based on the configured amount, then adjust this range if it means keeping
too many segments around, however we still must respect the required segments"
- It's worth updating the {{dfs.namenode.num.extra.edits.retained}} section in hdfs-default.xml
to indicate that the configured number of edits to retained may not be respected if it means
the total number of segments exceeds {{dfs.namenode.max.extra.edits.segments.retained}}.

Agree TestHDFSFileSystemContract looks spurious, filed HDFS-4027 so we can better see what's
going on there.
                
> HA: Put a cap on the number of completed edits files retained by the NN
> -----------------------------------------------------------------------
>
>                 Key: HDFS-2946
>                 URL: https://issues.apache.org/jira/browse/HDFS-2946
>             Project: Hadoop HDFS
>          Issue Type: Improvement
>          Components: ha, name-node
>    Affects Versions: 2.0.1-alpha
>            Reporter: Aaron T. Myers
>            Assignee: Aaron T. Myers
>         Attachments: HDFS-2946.patch
>
>
> HDFS-2794 added a minimum number of transactions to retain in edits files. Since many
underlying file systems put a cap on the number of entries in a single directory, we should
put a cap on the number of edits files which will be retained by the NN.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message