hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hadoop QA (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-3835) Long-lived 2NN cannot perform a checkpoint if security is enabled and the NN restarts without outstanding delegation tokens
Date Wed, 22 Aug 2012 05:28:38 GMT

    [ https://issues.apache.org/jira/browse/HDFS-3835?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13439288#comment-13439288
] 

Hadoop QA commented on HDFS-3835:
---------------------------------

-1 overall.  Here are the results of testing the latest attachment 
  http://issues.apache.org/jira/secure/attachment/12541851/HDFS-3835.patch
  against trunk revision .

    +1 @author.  The patch does not contain any @author tags.

    +1 tests included.  The patch appears to include 1 new or modified test files.

    +1 javac.  The applied patch does not increase the total number of javac compiler warnings.

    +1 javadoc.  The javadoc tool did not generate any warning messages.

    +1 eclipse:eclipse.  The patch built with eclipse:eclipse.

    -1 findbugs.  The patch appears to introduce 1 new Findbugs (version 1.3.9) warnings.

    +1 release audit.  The applied patch does not increase the total number of release audit
warnings.

    -1 core tests.  The patch failed these unit tests in hadoop-common-project/hadoop-common
hadoop-hdfs-project/hadoop-hdfs:

                  org.apache.hadoop.ha.TestZKFailoverController
                  org.apache.hadoop.hdfs.TestPersistBlocks

    +1 contrib tests.  The patch passed contrib unit tests.

Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/3065//testReport/
Findbugs warnings: https://builds.apache.org/job/PreCommit-HDFS-Build/3065//artifact/trunk/patchprocess/newPatchFindbugsWarningshadoop-hdfs.html
Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/3065//console

This message is automatically generated.
                
> Long-lived 2NN cannot perform a checkpoint if security is enabled and the NN restarts
without outstanding delegation tokens
> ---------------------------------------------------------------------------------------------------------------------------
>
>                 Key: HDFS-3835
>                 URL: https://issues.apache.org/jira/browse/HDFS-3835
>             Project: Hadoop HDFS
>          Issue Type: Bug
>          Components: name-node, security
>    Affects Versions: 2.0.0-alpha
>            Reporter: Aaron T. Myers
>            Assignee: Aaron T. Myers
>         Attachments: HDFS-3835.patch
>
>
> When the 2NN wants to perform a checkpoint, it figures out the highest transaction ID
of the fsimage files on the NN, and if the 2NN has a copy of that fsimage file (because it
created that merged fsimage file the last time it did a checkpoint) then the 2NN won't download
the fsimage file from the NN, and instead only gets the new edits files from the NN. In this
case, the 2NN also doesn't even bother reloading the fsimage file it has from disk, since
it has all of the namespace state in-memory. This all works just fine.
> When the 2NN _doesn't_ have a copy of the relevant fsimage file (for example, if the
NN had restarted since the last checkpoint) then the 2NN blows away its in-memory namespace
state, downloads the fsimage file from the NN, and loads the newly-downloaded fsimage file
from disk. The bug is that when the 2NN clears its in-memory state, it only resets the namespace,
but not the delegation token map.
> The fix is pretty simple - just make the delegation token map get cleared as well as
the namespace state when a running 2NN needs to load a new fsimage from disk.
> Credit to Stephen Chu for identifying this issue.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message