hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Daryn Sharp (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-4477) Secondary namenode may retain old tokens
Date Wed, 10 Apr 2013 16:36:15 GMT

    [ https://issues.apache.org/jira/browse/HDFS-4477?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13627960#comment-13627960
] 

Daryn Sharp commented on HDFS-4477:
-----------------------------------

Regarding earlier concern about the extra edits:  If the RM does its job correctly then tokens
will be cancelled when the job completes - which means we've already taken the hit for the
edit - and the NN won't have to issue internal cancels for expired tokens.  However, a "hack"
for oozie/pig causes the RM to never cancel tokens for those jobs.  After the hack is removed
(YARN-503 is the start), the NN should rarely have to expire tokens itself.
                
> Secondary namenode may retain old tokens
> ----------------------------------------
>
>                 Key: HDFS-4477
>                 URL: https://issues.apache.org/jira/browse/HDFS-4477
>             Project: Hadoop HDFS
>          Issue Type: Bug
>          Components: security
>    Affects Versions: 0.23.0, 2.0.0-alpha, 3.0.0
>            Reporter: Kihwal Lee
>            Assignee: Daryn Sharp
>            Priority: Critical
>         Attachments: HDFS-4477.patch, HDFS-4477.patch, HDFS-4477.patch
>
>
> Upon inspection of a fsimage created by a secondary namenode, we've discovered it contains
very old tokens. These are probably the ones that were not explicitly canceled.  It may be
related to the optimization done to avoid loading fsimage from scratch every time checkpointing.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message