hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "zhaojianbo (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HADOOP-13069) Making hadoop client proxy multi user in a process context
Date Fri, 29 Apr 2016 01:16:12 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-13069?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15263363#comment-15263363
] 

zhaojianbo commented on HADOOP-13069:
-------------------------------------

In our case, we want to proxy different users to use different resources(eg. different resource
pools in YARN, and different directories in HDFS with different permission) in one process
context. So we extend some methods int class UserGroupInformation to make it accept a parameter
"dfs.client.config.proxyuser" in conf which specify the user name needed. The code added like
this:
{code:java}
Index: hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/security/UserGroupInformation.java
===================================================================
--- hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/security/UserGroupInformation.java
(revision 2303)
+++ hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/security/UserGroupInformation.java
(working copy)
@@ -18,6 +18,7 @@
 package org.apache.hadoop.security;
 
 import static org.apache.hadoop.fs.CommonConfigurationKeys.HADOOP_USER_GROUP_METRICS_PERCENTILES_INTERVALS;
+import static org.apache.hadoop.fs.CommonConfigurationKeys.DFS_CLIENT_CONFIG_PROXYUSER;
 import static org.apache.hadoop.util.PlatformName.IBM_JAVA;
 
 import java.io.File;
@@ -342,6 +343,7 @@
    * Information about the logged in user.
    */
   private static UserGroupInformation loginUser = null;
+  private static HashMap<String, UserGroupInformation> map = new HashMap<String,UserGroupInformation>();
   private static String keytabPrincipal = null;
   private static String keytabFile = null;
 
@@ -650,6 +652,17 @@
     }
   }
 
+  public synchronized
+  static UserGroupInformation getCurrentUser(Configuration conf) throws IOException {
+    AccessControlContext context = AccessController.getContext();
+    Subject subject = Subject.getSubject(context);
+    if (subject == null || subject.getPrincipals(User.class).isEmpty()) {
+      return getLoginUser(conf);
+    } else {
+      return new UserGroupInformation(subject);
+    }
+  }
+
   /**
    * Find the most appropriate UserGroupInformation to use
    *
@@ -776,6 +789,20 @@
     return loginUser;
   }
 
+  public synchronized
+  static UserGroupInformation getLoginUser(Configuration conf) throws IOException {
+    String key = "";
+    synchronized (map){
+      key = conf.get(DFS_CLIENT_CONFIG_PROXYUSER,"");
+      if(map.containsKey(key)){
+        return map.get(key);
+      }
+    }
+    loginUserFromSubject(null,conf);
+    return map.get(key);
+  }
+
+
   /**
    * remove the login method that is followed by a space from the username
    * e.g. "jack (auth:SIMPLE)" -> "jack"
@@ -841,6 +868,55 @@
     } 
   }
 
+  public synchronized
+  static void loginUserFromSubject(Subject subject, Configuration conf) throws IOException
{
+    ensureInitialized();
+    UserGroupInformation tmpuser = null;
+    try {
+      if (subject == null) {
+        subject = new Subject();
+      }
+      LoginContext login =
+          newLoginContext(authenticationMethod.getLoginAppName(),
+                          subject, new HadoopConfiguration());
+      login.login();
+      UserGroupInformation realUser = new UserGroupInformation(subject);
+      realUser.setLogin(login);
+      realUser.setAuthenticationMethod(authenticationMethod);
+      realUser = new UserGroupInformation(login.getSubject());
+      // If the HADOOP_PROXY_USER environment variable or property
+      // is specified, create a proxy user as the logged in user.
+      String proxyUser = conf.get(DFS_CLIENT_CONFIG_PROXYUSER, "");
+      if (proxyUser==null || proxyUser.isEmpty()) {
+        proxyUser = System.getenv(HADOOP_PROXY_USER);
+        if (proxyUser == null) {
+          proxyUser = System.getProperty(HADOOP_PROXY_USER);
+        }
+      }
+      tmpuser = proxyUser == null ? realUser : createProxyUser(proxyUser, realUser);
+
+      String fileLocation = System.getenv(HADOOP_TOKEN_FILE_LOCATION);
+      if (fileLocation != null) {
+        // Load the token storage file and put all of the tokens into the
+        // user. Don't use the FileSystem API for reading since it has a lock
+        // cycle (HADOOP-9212).
+        Credentials cred = Credentials.readTokenStorageFile(
+            new File(fileLocation), conf);
+        tmpuser.addCredentials(cred);
+      }
+      tmpuser.spawnAutoRenewalThreadForUserCreds();
+      synchronized (map){
+        map.put(conf.get(DFS_CLIENT_CONFIG_PROXYUSER, ""), tmpuser);
+      }
+    } catch (LoginException le) {
+      LOG.debug("failure to login", le);
+      throw new IOException("failure to login", le);
+    }
+    if (LOG.isDebugEnabled()) {
+      LOG.debug("UGI loginUser:" + tmpuser);
+    } 
+  }
+
   @InterfaceAudience.Private
   @InterfaceStability.Unstable
   @VisibleForTesting
Index: hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/CommonConfigurationKeys.java
===================================================================
--- hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/CommonConfigurationKeys.java
(revision 2303)
+++ hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/CommonConfigurationKeys.java
(working copy)
@@ -281,4 +281,5 @@
   public static final String NFS_EXPORTS_ALLOWED_HOSTS_SEPARATOR = ";";
   public static final String NFS_EXPORTS_ALLOWED_HOSTS_KEY = "nfs.exports.allowed.hosts";
   public static final String NFS_EXPORTS_ALLOWED_HOSTS_KEY_DEFAULT = "* rw";
+  public static final String DFS_CLIENT_CONFIG_PROXYUSER = "dfs.client.config.proxyuser";
 }

{code}
And changing method getCurrentUser called into method getCurrentUser(conf) when needing the
feature.

> Making hadoop client proxy multi user in a process context
> ----------------------------------------------------------
>
>                 Key: HADOOP-13069
>                 URL: https://issues.apache.org/jira/browse/HADOOP-13069
>             Project: Hadoop Common
>          Issue Type: Improvement
>    Affects Versions: 2.6.0
>            Reporter: zhaojianbo
>
> Now hadoop client can proxy an other user if setting HADOOP_PROXY_USER in the environment
parameters. But the mechanism can proxy only one user in a process context. The code is like:
> {code:xml}
>   public synchronized 
>   static UserGroupInformation getLoginUser() throws IOException {
>     if (loginUser == null) {
>       loginUserFromSubject(null);
>     }
>     return loginUser;
>   }
> {code}
> The static variable loginUser cache the last login user.
> But in some cases, we want to proxy multi users in one process context. It seems to be
that the current code is not support that.Is it?
> Correct me if I'm wrong. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org


Mime
View raw message