hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Todd Lipcon (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-988) saveNamespace can corrupt edits log, apparently due to race conditions
Date Wed, 01 Jun 2011 01:48:47 GMT

    [ https://issues.apache.org/jira/browse/HDFS-988?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13041934#comment-13041934

Todd Lipcon commented on HDFS-988:

Didn't go through the new tests yet, but here are some comments from a first pass through

- checks for if (auditLog.isInfoEnabled()) should probably now be (auditLog.isInfoEnabled()
&& isExternalInvocation()) -- otherwise we're doing a needless directory traversal
for fsck
- The following methods currently do logSync() while holding the writeLock, which is expensive:
-- setPermission 
-- setOwner
-- commitBlockSynchronization (in some exit paths)
-- updatePipeline
- These methods should probably just set a local boolean within the synchronized section,
then logSync() in the finally clause if it's flagged

- seems strange that some of the xInternal() methods take the write lock themselves (eg setReplicationInternal)
whereas others assume the caller takes the write lock (eg createSymlinkInternal). We should
be consistent

- for those methods that don't explicitly take the write lock, we should either add an {{assert
hasWriteLock()}} or a comment explaining why the lock is not necessary (eg internalReleaseLease,
reassignLease, finalizeINodeFileUnderConstruction)

- why doesn't getListing need the read lock?
- comment for endCheckpoint says "not started" but should say "not ended"
- same with updatePipeline

- I noticed that nextGenerationStamp() doesn't logSync() -- that seems dangerous, since after
a restart we might hand out a duplicate genstamp.

> saveNamespace can corrupt edits log, apparently due to race conditions
> ----------------------------------------------------------------------
>                 Key: HDFS-988
>                 URL: https://issues.apache.org/jira/browse/HDFS-988
>             Project: Hadoop HDFS
>          Issue Type: Bug
>          Components: name-node
>    Affects Versions: 0.20-append, 0.21.0, 0.22.0
>            Reporter: dhruba borthakur
>            Assignee: Eli Collins
>            Priority: Blocker
>             Fix For: 0.20-append, 0.22.0
>         Attachments: HDFS-988_fix_synchs.patch, hdfs-988-2.patch, hdfs-988-3.patch, hdfs-988-4.patch,
hdfs-988.txt, saveNamespace.txt, saveNamespace_20-append.patch
> The adminstrator puts the namenode is safemode and then issues the savenamespace command.
This can corrupt the edits log. The problem is that  when the NN enters safemode, there could
still be pending logSycs occuring from other threads. Now, the saveNamespace command, when
executed, would save a edits log with partial writes. I have seen this happen on 0.20.
> https://issues.apache.org/jira/browse/HDFS-909?focusedCommentId=12828853&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12828853

This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

View raw message