hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Daryn Sharp (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-4477) Secondary namenode may retain old tokens
Date Tue, 26 Feb 2013 18:22:13 GMT

    [ https://issues.apache.org/jira/browse/HDFS-4477?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13587347#comment-13587347
] 

Daryn Sharp commented on HDFS-4477:
-----------------------------------

Thinking it through, I have a few concerns for consideration.  It's not as easy as starting
the TSM thread because it not only removes expired tokens but also starts generating and rolling
secret keys - which we definitely don't want.  There would have to be a way to enable/disable
secret key management independent of token expiration.

I'm a bit nervous about:
* Getting the enabling/disabling of token expiration and secret key management right for the
primary NN, 2NN, backup NN, and standby NNs.  Some would have slightly unique properties.
* Will also have to be careful to not break the JHS and RM that also use the ADTSM.  Changes
to secret key management cannot impact them.
* The 2NN changes from being a "dumb" data vessel, that only does what's it's told, to mutating
state.  If the config values don't match the primary NN, the 2NN may start expiring tokens
or secret keys too early.
* The offline image/edit viewers may be misleading.  If I want to see what should be the state
of the NN, for instance to verify this jira works, it will claim the NN is retaining tokens/secrets
that it's not.

I think writing expiration to the edit logs is a low-risk change because it simply parallels
an explicit cancellation and avoids the concerns cited above.  However I'm open to alternatives.
 Further thoughts?
                
> Secondary namenode may retain old tokens
> ----------------------------------------
>
>                 Key: HDFS-4477
>                 URL: https://issues.apache.org/jira/browse/HDFS-4477
>             Project: Hadoop HDFS
>          Issue Type: Bug
>          Components: security
>    Affects Versions: 0.23.0, 2.0.0-alpha, 3.0.0
>            Reporter: Kihwal Lee
>            Assignee: Daryn Sharp
>            Priority: Critical
>         Attachments: HDFS-4477.patch, HDFS-4477.patch
>
>
> Upon inspection of a fsimage created by a secondary namenode, we've discovered it contains
very old tokens. These are probably the ones that were not explicitly canceled.  It may be
related to the optimization done to avoid loading fsimage from scratch every time checkpointing.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message