Return-Path: 
 <yarn-issues-return-50481-apmail-hadoop-yarn-issues-archive=hadoop.apache.org@hadoop.apache.org>
X-Original-To: apmail-hadoop-yarn-issues-archive@minotaur.apache.org
Delivered-To: apmail-hadoop-yarn-issues-archive@minotaur.apache.org
Received: from mail.apache.org (hermes.apache.org [140.211.11.3])
	by minotaur.apache.org (Postfix) with SMTP id A099A177A6
	for <apmail-hadoop-yarn-issues-archive@minotaur.apache.org>;
 Thu, 12 Mar 2015 00:45:44 +0000 (UTC)
Received: (qmail 54540 invoked by uid 500); 12 Mar 2015 00:45:38 -0000
Delivered-To: apmail-hadoop-yarn-issues-archive@hadoop.apache.org
Received: (qmail 54477 invoked by uid 500); 12 Mar 2015 00:45:38 -0000
Mailing-List: contact yarn-issues-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
List-Help: <mailto:yarn-issues-help@hadoop.apache.org>
List-Unsubscribe: <mailto:yarn-issues-unsubscribe@hadoop.apache.org>
List-Post: <mailto:yarn-issues@hadoop.apache.org>
List-Id: <yarn-issues.hadoop.apache.org>
Reply-To: yarn-issues@hadoop.apache.org
Delivered-To: mailing list yarn-issues@hadoop.apache.org
Received: (qmail 54462 invoked by uid 99); 12 Mar 2015 00:45:38 -0000
Received: from arcas.apache.org (HELO arcas.apache.org) (140.211.11.28)
    by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 12 Mar 2015 00:45:38 +0000
Date: Thu, 12 Mar 2015 00:45:38 +0000 (UTC)
From: "zhihai xu (JIRA)" <jira@apache.org>
To: yarn-issues@hadoop.apache.org
Message-ID: <JIRA.12781372.1426121109000.70466.1426121138124@Atlassian.JIRA>
In-Reply-To: <JIRA.12781372.1426121109000@Atlassian.JIRA>
References: <JIRA.12781372.1426121109000@Atlassian.JIRA>
 <JIRA.12781372.1426121109854@arcas>
Subject: [jira] [Created] (YARN-3336) FileSystem memory leak in
 DelegationTokenRenewer
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit
X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394

zhihai xu created YARN-3336:
-------------------------------

             Summary: FileSystem memory leak in DelegationTokenRenewer
                 Key: YARN-3336
                 URL: https://issues.apache.org/jira/browse/YARN-3336
             Project: Hadoop YARN
          Issue Type: Bug
          Components: resourcemanager
            Reporter: zhihai xu
            Assignee: zhihai xu
            Priority: Critical


FileSystem memory leak in DelegationTokenRenewer.
Every time DelegationTokenRenewer#obtainSystemTokensForUser is called, a new FileSystem entry will be added to  FileSystem#CACHE which will never be garbage collected.
This is the implementation of obtainSystemTokensForUser:
{code}
  protected Token<?>[] obtainSystemTokensForUser(String user,
      final Credentials credentials) throws IOException, InterruptedException {
    // Get new hdfs tokens on behalf of this user
    UserGroupInformation proxyUser =
        UserGroupInformation.createProxyUser(user,
          UserGroupInformation.getLoginUser());
    Token<?>[] newTokens =
        proxyUser.doAs(new PrivilegedExceptionAction<Token<?>[]>() {
          @Override
          public Token<?>[] run() throws Exception {
            return FileSystem.get(getConfig()).addDelegationTokens(
              UserGroupInformation.getLoginUser().getUserName(), credentials);
          }
        });
    return newTokens;
  }
{code}

The memory leak happened when FileSystem.get(getConfig()) is called with a new proxy user.
Because createProxyUser will always create a new Subject.
{code}
public static UserGroupInformation createProxyUser(String user,
      UserGroupInformation realUser) {
    if (user == null || user.isEmpty()) {
      throw new IllegalArgumentException("Null user");
    }
    if (realUser == null) {
      throw new IllegalArgumentException("Null real user");
    }
    Subject subject = new Subject();
    Set<Principal> principals = subject.getPrincipals();
    principals.add(new User(user));
    principals.add(new RealUser(realUser));
    UserGroupInformation result =new UserGroupInformation(subject);
    result.setAuthenticationMethod(AuthenticationMethod.PROXY);
    return result;
  }
{code}

FileSystem#Cache#Key.equals will compare the ugi
{code}
      Key(URI uri, Configuration conf, long unique) throws IOException {
        scheme = uri.getScheme()==null?"":uri.getScheme().toLowerCase();
        authority = uri.getAuthority()==null?"":uri.getAuthority().toLowerCase();
        this.unique = unique;
        this.ugi = UserGroupInformation.getCurrentUser();
      }
      public boolean equals(Object obj) {
        if (obj == this) {
          return true;
        }
        if (obj != null && obj instanceof Key) {
          Key that = (Key)obj;
          return isEqual(this.scheme, that.scheme)
                 && isEqual(this.authority, that.authority)
                 && isEqual(this.ugi, that.ugi)
                 && (this.unique == that.unique);
        }
        return false;        
      }
{code}

UserGroupInformation.equals will compare subject by reference.
{code}
  public boolean equals(Object o) {
    if (o == this) {
      return true;
    } else if (o == null || getClass() != o.getClass()) {
      return false;
    } else {
      return subject == ((UserGroupInformation) o).subject;
    }
  }
{code}

So in this case, every time createProxyUser and FileSystem.get(getConfig()) are called, a new FileSystem will be created and a new entry will be added to FileSystem.CACHE.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)