hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Tsz Wo (Nicholas), SZE (Commented) (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-2589) unnecessary hftp token fetch and renewal thread
Date Wed, 14 Dec 2011 01:34:30 GMT

    [ https://issues.apache.org/jira/browse/HDFS-2589?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13168960#comment-13168960

Tsz Wo (Nicholas), SZE commented on HDFS-2589:

Hi Daryn, 

It seems that it is better to change HftpFileSystem but not DelegationTokenRenewer.

In WebHdfsFileSystem, the lazy initialization is implemented in addRenewAction(..), which
is the only method using DT_RENEWER.  It is more lazy -- we even don't have to create a DelegationTokenRenewer
object if addRenewAction(..) is not invoked.
  private static DelegationTokenRenewer<WebHdfsFileSystem> DT_RENEWER = null;

  private static synchronized void addRenewAction(final WebHdfsFileSystem webhdfs) {
    if (DT_RENEWER == null) {
      DT_RENEWER = new DelegationTokenRenewer<WebHdfsFileSystem>(WebHdfsFileSystem.class);

> unnecessary hftp token fetch and renewal thread
> -----------------------------------------------
>                 Key: HDFS-2589
>                 URL: https://issues.apache.org/jira/browse/HDFS-2589
>             Project: Hadoop HDFS
>          Issue Type: Bug
>          Components: security
>    Affects Versions: 1.0.0
>            Reporter: Daryn Sharp
>            Assignee: Daryn Sharp
>         Attachments: HDFS-2589.patch
> Instantiation of the hftp filesystem is causing a token to be implicitly created and
added to a custom token renewal thread.  With the new token renewal feature in the JT, this
causes the mapreduce {{obtainTokensForNamenodes}} to fetch two tokens (an implicit and uncancelled
token, and an explicit token) and leave a spurious renewal thread running.  This thread should
not be running in the JT.
> After speaking with Owen, the quick solution is to lazy fetch the token, and to lazy
start the renewer thread.

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira


View raw message