zeppelin-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From prabhjyotsi...@apache.org
Subject zeppelin git commit: [Zeppelin 2367] Hive JDBC proxy user option should be available even without kerberos
Date Wed, 03 May 2017 15:47:23 GMT
Repository: zeppelin
Updated Branches:
  refs/heads/master 62aec9fbd -> 08a981403


[Zeppelin 2367] Hive JDBC proxy user option should be available even without kerberos

### What is this PR for?
Hive JDBC proxy user option should be available generically.

### What type of PR is it?
[Improvement]

### What is the Jira issue?
* [Zeppelin 2367](https://issues.apache.org/jira/browse/ZEPPELIN-2367)

### How should this be tested?
Enable Shiro authentication and set `zeppelin.jdbc.auth.type` as `SIMPLE` in the interpreter
setting, and observe the connection string for the Hive.

### Screenshots (if appropriate)

### Questions:
* Does the licenses files need an update? N/A
* Is there breaking changes for older versions? N/A
* Does this needs documentation? Yes

Author: Prabhjyot Singh <prabhjyotsingh@gmail.com>

Closes #2229 from prabhjyotsingh/ZEPPELIN-2367 and squashes the following commits:

84b5e55b2 [Prabhjyot Singh] add logger.warn for hive and impersonation
45c90a8e2 [Prabhjyot Singh] improve doc
9fee9d2a9 [Prabhjyot Singh] replace hive with generic method
a348e969a [Prabhjyot Singh] revert "zeppelin.jdbc.auth.kerberos.proxy.enable" behaviour
e2bdbb2ad [Prabhjyot Singh] include e as inner exception
c180f5ce2 [Prabhjyot Singh] Merge remote-tracking branch 'origin/master' into ZEPPELIN-2367
1802b453f [Prabhjyot Singh] remove hive string from logger
513987a28 [Prabhjyot Singh] apply genric logic to appendProxyUserToURL
3fa2b1e98 [Prabhjyot Singh] change name to appendProxyUserToURL
a75167415 [Prabhjyot Singh] Merge remote-tracking branch 'origin/master' into ZEPPELIN-2367
4c382eefa [Prabhjyot Singh] log user details as well
d51e770b2 [Prabhjyot Singh] add doc in jdbc.md
01b18b9d2 [Prabhjyot Singh] add doc (reverted from commit ee8a6b524c481210486761032cb1f5fd6266bb54)
40489c89d [Prabhjyot Singh] Merge remote-tracking branch 'origin/master' into ZEPPELIN-2367
ee8a6b524 [Prabhjyot Singh] add doc
8999d93ae [Prabhjyot Singh] ZEPPELIN-2367: Hive JDBC proxy user option should be avail even
without kerberos


Project: http://git-wip-us.apache.org/repos/asf/zeppelin/repo
Commit: http://git-wip-us.apache.org/repos/asf/zeppelin/commit/08a98140
Tree: http://git-wip-us.apache.org/repos/asf/zeppelin/tree/08a98140
Diff: http://git-wip-us.apache.org/repos/asf/zeppelin/diff/08a98140

Branch: refs/heads/master
Commit: 08a9814033e216eb2d340c49d171f3b66c19eb61
Parents: 62aec9f
Author: Prabhjyot Singh <prabhjyotsingh@gmail.com>
Authored: Sun Apr 30 00:15:19 2017 +0530
Committer: Prabhjyot Singh <prabhjyotsingh@gmail.com>
Committed: Wed May 3 21:17:16 2017 +0530

----------------------------------------------------------------------
 docs/interpreter/jdbc.md                        | 35 ++++++++---
 .../apache/zeppelin/jdbc/JDBCInterpreter.java   | 66 +++++++++++---------
 .../jdbc/security/JDBCSecurityImpl.java         |  5 +-
 .../interpreter/InterpreterException.java       |  4 ++
 4 files changed, 68 insertions(+), 42 deletions(-)
----------------------------------------------------------------------


http://git-wip-us.apache.org/repos/asf/zeppelin/blob/08a98140/docs/interpreter/jdbc.md
----------------------------------------------------------------------
diff --git a/docs/interpreter/jdbc.md b/docs/interpreter/jdbc.md
index f66ad6d..f10444b 100644
--- a/docs/interpreter/jdbc.md
+++ b/docs/interpreter/jdbc.md
@@ -171,7 +171,7 @@ There are more JDBC interpreter properties you can specify like below.
   </tr>
   <tr>
       <td>zeppelin.jdbc.auth.kerberos.proxy.enable</td>
-      <td>When auth type is Kerberos, enable/disable Kerberos proxy with the login
user to get the connection. Default value is true.</td>
+      <td>When auth type is Kerberos, enable/disable Kerberos proxy with the login
user to get the connection. Default value is true.</td>
   </tr>
   <tr>
     <td>default.jceks.file</td>
@@ -202,7 +202,7 @@ To bind the interpreters created in the interpreter setting page, click
the gear
 
 <img src="../assets/themes/zeppelin/img/docs-img/click_interpreter_binding_button.png"
width="600px" />
 
-Select(blue) or deselect(white) the interpreter buttons depending on your use cases. 
+Select(blue) or deselect(white) the interpreter buttons depending on your use cases.
 If you need to use more than one interpreter in the notebook, activate several buttons.
 Don't forget to click `Save` button, or you will face `Interpreter *** is not found` error.
 
@@ -285,7 +285,7 @@ An example settings of interpreter for the two data sources, each of which
has i
 </table>
 
 ##### Usage
-Test of execution *precode* for each data source. 
+Test of execution *precode* for each data source.
 
 ```sql
 %jdbc
@@ -480,7 +480,7 @@ Here are some examples you can refer to. Including the below connectors,
you can
 
 [Maven Repository: com.amazonaws:aws-java-sdk-redshift](https://mvnrepository.com/artifact/com.amazonaws/aws-java-sdk-redshift)
 
-### Apache Hive 
+### Apache Hive
 
 <img src="../assets/themes/zeppelin/img/docs-img/hive_setting.png" width="600px" />
 
@@ -507,12 +507,11 @@ Here are some examples you can refer to. Including the below connectors,
you can
     <td>hive_password</td>
   </tr>
   <tr>
-    <td>hive.proxy.user</td>
-    <td>true or false</td>
+    <td>default.proxy.user.property</td>
+    <td>Example value: hive.server2.proxy.user</td>
+  </tr>
 </table>
 
-Connection to Hive JDBC with a proxy user can be disabled with `hive.proxy.user` property
(set to true by default)
-
 [Apache Hive 1 JDBC Driver Docs](https://cwiki.apache.org/confluence/display/Hive/HiveServer2+Clients#HiveServer2Clients-JDBC)
 [Apache Hive 2 JDBC Driver Docs](https://cwiki.apache.org/confluence/display/Hive/HiveServer2+Clients#HiveServer2Clients-JDBC)
 
@@ -534,6 +533,26 @@ Connection to Hive JDBC with a proxy user can be disabled with `hive.proxy.user`
 
 [Maven Repository : org.apache.hive:hive-jdbc](https://mvnrepository.com/artifact/org.apache.hive/hive-jdbc)
 
+##### Impersonation
+When Zeppelin server is running with authentication enabled, then the interpreter can utilize
Hive's user proxy feature i.e. send extra parameter for creating and running a session ("hive.server2.proxy.user=":
"${loggedInUser}"). This is particularly useful when multiple users are sharing a notebooks.
+
+To enable this set following:
+  - `zeppelin.jdbc.auth.type` as `SIMPLE` or `KERBEROS` (if required) in the interpreter
setting.
+  - `${prefix}.proxy.user.property` as `hive.server2.proxy.user`
+  Example configuration
+
+  *Properties*
+
+  | name                      | value                                                   
                                         |
+  |:------------------------- |:--------------------------------------------------------------------------------------------------|
+  | hive.driver               | org.apache.hive.jdbc.HiveDriver                         
                                         |
+  | hive.password             |                                                         
                                         |
+  | hive.url                  | jdbc:hive2://hive-server-host:2181/;serviceDiscoveryMode=zooKeeper;zooKeeperNamespace=hiveserver2
|
+  | hive.proxy.user.property  | hive.proxy.user.property                                
                                         |
+  | zeppelin.jdbc.auth.type   | SIMPLE                                                  
                                         |
+
+
+
 ### Apache Phoenix
 
 Phoenix supports `thick` and `thin` connection types:

http://git-wip-us.apache.org/repos/asf/zeppelin/blob/08a98140/jdbc/src/main/java/org/apache/zeppelin/jdbc/JDBCInterpreter.java
----------------------------------------------------------------------
diff --git a/jdbc/src/main/java/org/apache/zeppelin/jdbc/JDBCInterpreter.java b/jdbc/src/main/java/org/apache/zeppelin/jdbc/JDBCInterpreter.java
index 95b2a1a..3272bc1 100644
--- a/jdbc/src/main/java/org/apache/zeppelin/jdbc/JDBCInterpreter.java
+++ b/jdbc/src/main/java/org/apache/zeppelin/jdbc/JDBCInterpreter.java
@@ -26,7 +26,6 @@ import java.util.ArrayList;
 import java.util.HashMap;
 import java.util.HashSet;
 import java.util.List;
-import java.util.Map;
 import java.util.Properties;
 import java.util.Set;
 
@@ -179,10 +178,6 @@ public class JDBCInterpreter extends Interpreter {
     }
     logger.debug("JDBC PropretiesMap: {}", basePropretiesMap);
 
-    if (!isEmpty(property.getProperty("zeppelin.jdbc.auth.type"))) {
-      JDBCSecurityImpl.createSecureConfiguration(property);
-    }
-
     setMaxLineResults();
   }
 
@@ -355,36 +350,25 @@ public class JDBCInterpreter extends Interpreter {
     } else {
       UserGroupInformation.AuthenticationMethod authType = JDBCSecurityImpl.getAuthtype(property);
 
+      final String connectionUrl = appendProxyUserToURL(url, user, propertyKey);
+
+      JDBCSecurityImpl.createSecureConfiguration(property, authType);
       switch (authType) {
           case KERBEROS:
             if (user == null || "false".equalsIgnoreCase(
-              property.getProperty("zeppelin.jdbc.auth.kerberos.proxy.enable"))) {
-              connection = getConnectionFromPool(url, user, propertyKey, properties);
+                property.getProperty("zeppelin.jdbc.auth.kerberos.proxy.enable"))) {
+              connection = getConnectionFromPool(connectionUrl, user, propertyKey, properties);
             } else {
-              if (url.trim().startsWith("jdbc:hive")) {
-                StringBuilder connectionUrl = new StringBuilder(url);
-                Integer lastIndexOfUrl = connectionUrl.indexOf("?");
-                if (lastIndexOfUrl == -1) {
-                  lastIndexOfUrl = connectionUrl.length();
-                }
-                boolean hasProxyUser = property.containsKey("hive.proxy.user");
-                if (!hasProxyUser || !property.getProperty("hive.proxy.user").equals("false")){
-                  logger.debug("Using hive proxy user");
-                  connectionUrl.insert(lastIndexOfUrl, ";hive.server2.proxy.user=" + user
+ ";");
-                }
-                connection = getConnectionFromPool(connectionUrl.toString(),
-                        user, propertyKey, properties);
+              if (basePropretiesMap.get(propertyKey).containsKey("proxy.user.property"))
{
+                connection = getConnectionFromPool(connectionUrl, user, propertyKey, properties);
               } else {
                 UserGroupInformation ugi = null;
                 try {
                   ugi = UserGroupInformation.createProxyUser(
-                          user, UserGroupInformation.getCurrentUser());
+                      user, UserGroupInformation.getCurrentUser());
                 } catch (Exception e) {
                   logger.error("Error in getCurrentUser", e);
-                  StringBuilder stringBuilder = new StringBuilder();
-                  stringBuilder.append(e.getMessage()).append("\n");
-                  stringBuilder.append(e.getCause());
-                  throw new InterpreterException(stringBuilder.toString());
+                  throw new InterpreterException("Error in getCurrentUser", e);
                 }
 
                 final String poolKey = propertyKey;
@@ -392,28 +376,48 @@ public class JDBCInterpreter extends Interpreter {
                   connection = ugi.doAs(new PrivilegedExceptionAction<Connection>()
{
                     @Override
                     public Connection run() throws Exception {
-                      return getConnectionFromPool(url, user, poolKey, properties);
+                      return getConnectionFromPool(connectionUrl, user, poolKey, properties);
                     }
                   });
                 } catch (Exception e) {
                   logger.error("Error in doAs", e);
-                  StringBuilder stringBuilder = new StringBuilder();
-                  stringBuilder.append(e.getMessage()).append("\n");
-                  stringBuilder.append(e.getCause());
-                  throw new InterpreterException(stringBuilder.toString());
+                  throw new InterpreterException("Error in doAs", e);
                 }
               }
             }
             break;
 
           default:
-            connection = getConnectionFromPool(url, user, propertyKey, properties);
+            connection = getConnectionFromPool(connectionUrl, user, propertyKey, properties);
       }
     }
 
     return connection;
   }
 
+  private String appendProxyUserToURL(String url, String user, String propertyKey) {
+    StringBuilder connectionUrl = new StringBuilder(url);
+
+    if (user != null && !user.equals("anonymous") &&
+        basePropretiesMap.get(propertyKey).containsKey("proxy.user.property")) {
+
+      Integer lastIndexOfUrl = connectionUrl.indexOf("?");
+      if (lastIndexOfUrl == -1) {
+        lastIndexOfUrl = connectionUrl.length();
+      }
+      logger.info("Using proxy user as :" + user);
+      logger.info("Using proxy property for user as :" +
+          basePropretiesMap.get(propertyKey).getProperty("proxy.user.property"));
+      connectionUrl.insert(lastIndexOfUrl, ";" +
+          basePropretiesMap.get(propertyKey).getProperty("proxy.user.property") + "=" + user
+ ";");
+    } else if (user != null && !user.equals("anonymous") && url.contains("hive"))
{
+      logger.warn("User impersonation for hive has changed please refer: http://zeppelin.apache"
+
+          ".org/docs/latest/interpreter/jdbc.html#apache-hive");
+    }
+
+    return connectionUrl.toString();
+  }
+
   private String getPassword(Properties properties) throws IOException {
     if (isNotEmpty(properties.getProperty(PASSWORD_KEY))) {
       return properties.getProperty(PASSWORD_KEY);

http://git-wip-us.apache.org/repos/asf/zeppelin/blob/08a98140/jdbc/src/main/java/org/apache/zeppelin/jdbc/security/JDBCSecurityImpl.java
----------------------------------------------------------------------
diff --git a/jdbc/src/main/java/org/apache/zeppelin/jdbc/security/JDBCSecurityImpl.java b/jdbc/src/main/java/org/apache/zeppelin/jdbc/security/JDBCSecurityImpl.java
index 32a7990..25959e1 100644
--- a/jdbc/src/main/java/org/apache/zeppelin/jdbc/security/JDBCSecurityImpl.java
+++ b/jdbc/src/main/java/org/apache/zeppelin/jdbc/security/JDBCSecurityImpl.java
@@ -38,9 +38,8 @@ public class JDBCSecurityImpl {
   /***
    * @param properties
    */
-  public static void createSecureConfiguration(Properties properties) {
-    AuthenticationMethod authType = getAuthtype(properties);
-
+  public static void createSecureConfiguration(Properties properties,
+      AuthenticationMethod authType) {
     switch (authType) {
         case KERBEROS:
           Configuration conf = new

http://git-wip-us.apache.org/repos/asf/zeppelin/blob/08a98140/zeppelin-interpreter/src/main/java/org/apache/zeppelin/interpreter/InterpreterException.java
----------------------------------------------------------------------
diff --git a/zeppelin-interpreter/src/main/java/org/apache/zeppelin/interpreter/InterpreterException.java
b/zeppelin-interpreter/src/main/java/org/apache/zeppelin/interpreter/InterpreterException.java
index 30c1c0a..ebd184e 100644
--- a/zeppelin-interpreter/src/main/java/org/apache/zeppelin/interpreter/InterpreterException.java
+++ b/zeppelin-interpreter/src/main/java/org/apache/zeppelin/interpreter/InterpreterException.java
@@ -31,4 +31,8 @@ public class InterpreterException extends RuntimeException {
     super(m);
   }
 
+  public InterpreterException(String msg, Throwable t) {
+    super(msg, t);
+  }
+
 }


Mime
View raw message