hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hadoop QA (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-988) saveNamespace can corrupt edits log, apparently due to race conditions
Date Thu, 02 Jun 2011 04:24:47 GMT

    [ https://issues.apache.org/jira/browse/HDFS-988?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13042600#comment-13042600
] 

Hadoop QA commented on HDFS-988:
--------------------------------

-1 overall.  Here are the results of testing the latest attachment 
  http://issues.apache.org/jira/secure/attachment/12481173/hdfs-988-5.patch
  against trunk revision 1130339.

    +1 @author.  The patch does not contain any @author tags.

    +1 tests included.  The patch appears to include 18 new or modified tests.

    +1 javadoc.  The javadoc tool did not generate any warning messages.

    +1 javac.  The applied patch does not increase the total number of javac compiler warnings.

    +1 findbugs.  The patch does not introduce any new Findbugs (version 1.3.9) warnings.

    +1 release audit.  The applied patch does not increase the total number of release audit
warnings.

    -1 core tests.  The patch failed these core unit tests:
                  org.apache.hadoop.cli.TestHDFSCLI
                  org.apache.hadoop.hdfs.server.namenode.TestCheckPointForSecurityTokens
                  org.apache.hadoop.hdfs.server.namenode.TestCheckpoint
                  org.apache.hadoop.hdfs.server.namenode.TestEditLogRace
                  org.apache.hadoop.hdfs.server.namenode.TestParallelImageWrite
                  org.apache.hadoop.hdfs.server.namenode.TestSaveNamespace
                  org.apache.hadoop.hdfs.server.namenode.TestStartup
                  org.apache.hadoop.hdfs.TestDFSFinalize
                  org.apache.hadoop.hdfs.TestDFSRollback
                  org.apache.hadoop.hdfs.TestDFSStartupVersions
                  org.apache.hadoop.hdfs.TestDFSStorageStateRecovery
                  org.apache.hadoop.hdfs.TestDFSUpgradeFromImage
                  org.apache.hadoop.hdfs.TestDFSUpgrade
                  org.apache.hadoop.hdfs.TestListFilesInDFS
                  org.apache.hadoop.hdfs.TestListFilesInFileContext
                  org.apache.hadoop.hdfs.tools.offlineImageViewer.TestOfflineImageViewer

    +1 contrib tests.  The patch passed contrib unit tests.

    +1 system test framework.  The patch passed system test framework compile.

Test results: https://builds.apache.org/hudson/job/PreCommit-HDFS-Build/679//testReport/
Findbugs warnings: https://builds.apache.org/hudson/job/PreCommit-HDFS-Build/679//artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
Console output: https://builds.apache.org/hudson/job/PreCommit-HDFS-Build/679//console

This message is automatically generated.

> saveNamespace can corrupt edits log, apparently due to race conditions
> ----------------------------------------------------------------------
>
>                 Key: HDFS-988
>                 URL: https://issues.apache.org/jira/browse/HDFS-988
>             Project: Hadoop HDFS
>          Issue Type: Bug
>          Components: name-node
>    Affects Versions: 0.20-append, 0.21.0, 0.22.0
>            Reporter: dhruba borthakur
>            Assignee: Eli Collins
>            Priority: Blocker
>             Fix For: 0.20-append, 0.22.0
>
>         Attachments: HDFS-988_fix_synchs.patch, hdfs-988-2.patch, hdfs-988-3.patch, hdfs-988-4.patch,
hdfs-988-5.patch, hdfs-988-b22-1.patch, hdfs-988.txt, saveNamespace.txt, saveNamespace_20-append.patch
>
>
> The adminstrator puts the namenode is safemode and then issues the savenamespace command.
This can corrupt the edits log. The problem is that  when the NN enters safemode, there could
still be pending logSycs occuring from other threads. Now, the saveNamespace command, when
executed, would save a edits log with partial writes. I have seen this happen on 0.20.
> https://issues.apache.org/jira/browse/HDFS-909?focusedCommentId=12828853&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12828853

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message