hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Tsz Wo (Nicholas), SZE (Commented) (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-2589) unnecessary hftp token fetch and renewal thread
Date Wed, 14 Dec 2011 01:34:30 GMT

    [ https://issues.apache.org/jira/browse/HDFS-2589?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13168960#comment-13168960
] 

Tsz Wo (Nicholas), SZE commented on HDFS-2589:
----------------------------------------------

Hi Daryn, 

It seems that it is better to change HftpFileSystem but not DelegationTokenRenewer.

In WebHdfsFileSystem, the lazy initialization is implemented in addRenewAction(..), which
is the only method using DT_RENEWER.  It is more lazy -- we even don't have to create a DelegationTokenRenewer
object if addRenewAction(..) is not invoked.
{code}
//WebHdfsFileSystem
  private static DelegationTokenRenewer<WebHdfsFileSystem> DT_RENEWER = null;

  private static synchronized void addRenewAction(final WebHdfsFileSystem webhdfs) {
    if (DT_RENEWER == null) {
      DT_RENEWER = new DelegationTokenRenewer<WebHdfsFileSystem>(WebHdfsFileSystem.class);
      DT_RENEWER.start();
    }

    DT_RENEWER.addRenewAction(webhdfs);
  }
{code}
                
> unnecessary hftp token fetch and renewal thread
> -----------------------------------------------
>
>                 Key: HDFS-2589
>                 URL: https://issues.apache.org/jira/browse/HDFS-2589
>             Project: Hadoop HDFS
>          Issue Type: Bug
>          Components: security
>    Affects Versions: 1.0.0
>            Reporter: Daryn Sharp
>            Assignee: Daryn Sharp
>         Attachments: HDFS-2589.patch
>
>
> Instantiation of the hftp filesystem is causing a token to be implicitly created and
added to a custom token renewal thread.  With the new token renewal feature in the JT, this
causes the mapreduce {{obtainTokensForNamenodes}} to fetch two tokens (an implicit and uncancelled
token, and an explicit token) and leave a spurious renewal thread running.  This thread should
not be running in the JT.
> After speaking with Owen, the quick solution is to lazy fetch the token, and to lazy
start the renewer thread.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message