hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Todd Lipcon (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-988) saveNamespace can corrupt edits log
Date Wed, 18 May 2011 05:11:47 GMT

    [ https://issues.apache.org/jira/browse/HDFS-988?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13035201#comment-13035201
] 

Todd Lipcon commented on HDFS-988:
----------------------------------

Looking at trunk I see a lot of other strange synchronization issues. The following methods
all are synchronized on the FSNamesystem instance:
- getNamespaceInfo 
- setQuota 
- renewLease
- isInSafeMode
- isInStartupSafeMode
- isPopulatingReplQueues
- nextGenerationStamp

I think all of these should probably be using the new rwlock... Dhruba, what do you think?

Maybe we need something more like a stress/fuzz test against FSNamesystem rather than trying
to target the specific cases mentioned above?


> saveNamespace can corrupt edits log
> -----------------------------------
>
>                 Key: HDFS-988
>                 URL: https://issues.apache.org/jira/browse/HDFS-988
>             Project: Hadoop HDFS
>          Issue Type: Bug
>          Components: name-node
>    Affects Versions: 0.20-append, 0.21.0, 0.22.0
>            Reporter: dhruba borthakur
>            Assignee: Todd Lipcon
>            Priority: Blocker
>             Fix For: 0.20-append, 0.22.0
>
>         Attachments: hdfs-988-2.patch, hdfs-988.txt, saveNamespace.txt, saveNamespace_20-append.patch
>
>
> The adminstrator puts the namenode is safemode and then issues the savenamespace command.
This can corrupt the edits log. The problem is that  when the NN enters safemode, there could
still be pending logSycs occuring from other threads. Now, the saveNamespace command, when
executed, would save a edits log with partial writes. I have seen this happen on 0.20.
> https://issues.apache.org/jira/browse/HDFS-909?focusedCommentId=12828853&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12828853

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message