Mailing-List: contact hdfs-issues-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: hdfs-issues@hadoop.apache.org
Date: Tue, 26 Feb 2013 18:22:13 +0000 (UTC)
From: "Daryn Sharp (JIRA)" <jira@apache.org>
To: hdfs-issues@hadoop.apache.org
Message-ID: <JIRA.12631303.1360256507114.343538.1361902933637@arcas>
In-Reply-To: <JIRA.12631303.1360256507114@arcas>
References: <JIRA.12631303.1360256507114@arcas>
Subject: [jira] [Commented] (HDFS-4477) Secondary namenode may retain old
 tokens
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit


    [ https://issues.apache.org/jira/browse/HDFS-4477?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13587347#comment-13587347 ] 

Daryn Sharp commented on HDFS-4477:
-----------------------------------

Thinking it through, I have a few concerns for consideration.  It's not as easy as starting the TSM thread because it not only removes expired tokens but also starts generating and rolling secret keys - which we definitely don't want.  There would have to be a way to enable/disable secret key management independent of token expiration.

I'm a bit nervous about:
* Getting the enabling/disabling of token expiration and secret key management right for the primary NN, 2NN, backup NN, and standby NNs.  Some would have slightly unique properties.
* Will also have to be careful to not break the JHS and RM that also use the ADTSM.  Changes to secret key management cannot impact them.
* The 2NN changes from being a "dumb" data vessel, that only does what's it's told, to mutating state.  If the config values don't match the primary NN, the 2NN may start expiring tokens or secret keys too early.
* The offline image/edit viewers may be misleading.  If I want to see what should be the state of the NN, for instance to verify this jira works, it will claim the NN is retaining tokens/secrets that it's not.

I think writing expiration to the edit logs is a low-risk change because it simply parallels an explicit cancellation and avoids the concerns cited above.  However I'm open to alternatives.  Further thoughts?
                
> Secondary namenode may retain old tokens
> ----------------------------------------
>
>                 Key: HDFS-4477
>                 URL: https://issues.apache.org/jira/browse/HDFS-4477
>             Project: Hadoop HDFS
>          Issue Type: Bug
>          Components: security
>    Affects Versions: 0.23.0, 2.0.0-alpha, 3.0.0
>            Reporter: Kihwal Lee
>            Assignee: Daryn Sharp
>            Priority: Critical
>         Attachments: HDFS-4477.patch, HDFS-4477.patch
>
>
> Upon inspection of a fsimage created by a secondary namenode, we've discovered it contains very old tokens. These are probably the ones that were not explicitly canceled.  It may be related to the optimization done to avoid loading fsimage from scratch every time checkpointing.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira