hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hudson (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-4739) NN can miscalculate the number of extra edit log segments to retain
Date Thu, 25 Apr 2013 14:08:23 GMT

    [ https://issues.apache.org/jira/browse/HDFS-4739?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13641830#comment-13641830
] 

Hudson commented on HDFS-4739:
------------------------------

Integrated in Hadoop-Mapreduce-trunk #1410 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1410/])
    HDFS-4739. NN can miscalculate the number of extra edit log segments to retain. Contributed
by Aaron T. Myers. (Revision 1471769)

     Result = SUCCESS
atm : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1471769
Files : 
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NNStorageRetentionManager.java
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestNNStorageRetentionManager.java

                
> NN can miscalculate the number of extra edit log segments to retain
> -------------------------------------------------------------------
>
>                 Key: HDFS-4739
>                 URL: https://issues.apache.org/jira/browse/HDFS-4739
>             Project: Hadoop HDFS
>          Issue Type: Bug
>          Components: namenode
>    Affects Versions: 2.0.4-alpha
>            Reporter: Aaron T. Myers
>            Assignee: Aaron T. Myers
>             Fix For: 2.0.5-beta
>
>         Attachments: HDFS-4739.patch, HDFS-4739.patch
>
>
> The code in NNStorageRetentionManager#purgeOldStorage is intended to place a cap on the
number of _extra_ edit log segments retained beyond what is strictly required to replay the
FS history since the last fsimage. In fact this code currently places a limit on the _total_
number of extra edit log segments. If the number of required segments is greater than the
configured cap, there will be no data loss, but an ugly error will be thrown and the NN will
fail to start.
> The fix is simple, and in the meantime a work-around is just to raise the value of dfs.namenode.max.extra.edits.segments.retained
and start the NN.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message