zeppelin-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From felixche...@apache.org
Subject zeppelin git commit: [ZEPPELIN-1999] get interpreter property with replaced context parameters
Date Sat, 25 Mar 2017 21:54:17 GMT
Repository: zeppelin
Updated Branches:
  refs/heads/master a3c7f985f -> 998c8f35e


[ZEPPELIN-1999] get interpreter property with replaced context parameters

### What is this PR for?
Adds posibility to use context parameters (types: String.class, Double.class, Float.class,
Short.class,
Byte.class, Character.class, Boolean.class, Integer.class, Long.class, ) into property value
of interpreter.

### What type of PR is it?
Feature

### What is the Jira issue?
https://issues.apache.org/jira/browse/ZEPPELIN-1999

### How should this be tested?
1. Add text with markers #{contextFieldNAme} (ex. #{noteId} or #{replName}) to interpreter
property value (or add new property of interpreter).
2. Get this property (getProperty(key)), markers should be replaced by context values

### Questions:
* Does the licenses files need update? no
* Is there breaking changes for older versions? no
* Does this needs documentation? yes

Author: Tinkoff DWH <tinkoff.dwh@gmail.com>

Closes #2085 from tinkoff-dwh/ZEPPELIN-1999 and squashes the following commits:

fa1500a [Tinkoff DWH] [ZEPPELIN-1999] fix logic of replace
93c759d [Tinkoff DWH] Merge remote-tracking branch 'origin/master' into ZEPPELIN-1999
be4fada [Tinkoff DWH] [ZEPPELIN-1999] revert gitignore
c0110e9 [Tinkoff DWH] [ZEPPELIN-1999] documentation
61ac564 [Tinkoff DWH] [ZEPPELIN-1999] docs
a10dc0e [Tinkoff DWH] [ZEPPELIN-1999] skip fields of paragraph
ea9c6a3 [Tinkoff DWH] [ZEPPELIN-1999] docs
7c4489c [Tinkoff DWH] Merge remote-tracking branch 'origin/master' into ZEPPELIN-1999
527419a [Tinkoff DWH] Merge remote-tracking branch 'origin/master' into ZEPPELIN-1999
b5424b9 [Tinkoff DWH] [ZEPPELIN-1999] get interpreter property with replaced context parameters


Project: http://git-wip-us.apache.org/repos/asf/zeppelin/repo
Commit: http://git-wip-us.apache.org/repos/asf/zeppelin/commit/998c8f35
Tree: http://git-wip-us.apache.org/repos/asf/zeppelin/tree/998c8f35
Diff: http://git-wip-us.apache.org/repos/asf/zeppelin/diff/998c8f35

Branch: refs/heads/master
Commit: 998c8f35e894335ae757867982401e4ba4118d00
Parents: a3c7f98
Author: Tinkoff DWH <tinkoff.dwh@gmail.com>
Authored: Wed Mar 22 14:20:38 2017 +0500
Committer: Felix Cheung <felixcheung@apache.org>
Committed: Sat Mar 25 14:54:12 2017 -0700

----------------------------------------------------------------------
 ...erpreter_setting_with_context_parameters.png | Bin 0 -> 17290 bytes
 docs/manual/interpreters.md                     |  35 ++++++++++++++-
 .../zeppelin/interpreter/Interpreter.java       |  43 ++++++++++++++++++-
 .../zeppelin/interpreter/InterpreterTest.java   |  39 ++++++++++++++++-
 4 files changed, 112 insertions(+), 5 deletions(-)
----------------------------------------------------------------------


http://git-wip-us.apache.org/repos/asf/zeppelin/blob/998c8f35/docs/assets/themes/zeppelin/img/screenshots/interpreter_setting_with_context_parameters.png
----------------------------------------------------------------------
diff --git a/docs/assets/themes/zeppelin/img/screenshots/interpreter_setting_with_context_parameters.png
b/docs/assets/themes/zeppelin/img/screenshots/interpreter_setting_with_context_parameters.png
new file mode 100644
index 0000000..17c83b6
Binary files /dev/null and b/docs/assets/themes/zeppelin/img/screenshots/interpreter_setting_with_context_parameters.png
differ

http://git-wip-us.apache.org/repos/asf/zeppelin/blob/998c8f35/docs/manual/interpreters.md
----------------------------------------------------------------------
diff --git a/docs/manual/interpreters.md b/docs/manual/interpreters.md
index 70b94d4..916d591 100644
--- a/docs/manual/interpreters.md
+++ b/docs/manual/interpreters.md
@@ -41,8 +41,39 @@ Zeppelin interpreter setting is the configuration of a given interpreter
on Zepp
 
 <img src="../assets/themes/zeppelin/img/screenshots/interpreter_setting.png" width="500px">
 
-Properties are exported as environment variable when property name is consisted of upper
characters, numbers and underscore ([A-Z_0-9]). Otherwise set properties as JVM property.
-
+Properties are exported as environment variables when property name is consisted of upper
characters, numbers and underscore ([A-Z_0-9]). Otherwise set properties as JVM property.

+
+You may use parameters from the context of interpreter by add #{contextParameterName} in
value, parameter can be of the following types: string, number, boolean.
+
+###### Context parameters
+<table class="table-configuration">
+  <tr>
+    <th>Name</th>
+    <th>Type</th>
+  </tr>
+  <tr>
+    <td>user</td>
+    <td>string</td>
+  </tr>
+  <tr>
+    <td>noteId</td>
+    <td>string</td>
+  </tr>
+  <tr>
+    <td>replName</td>
+    <td>string</td>
+  </tr>
+  <tr>
+    <td>className</td>
+    <td>string</td>
+  </tr>
+</table>
+
+If context parameter is null then replaced by empty string.
+
+<img src="../assets/themes/zeppelin/img/screenshots/interpreter_setting_with_context_parameters.png"
width="800px">
+
+<br>
 Each notebook can be bound to multiple Interpreter Settings using setting icon on upper right
corner of the notebook.
 
 <img src="../assets/themes/zeppelin/img/screenshots/interpreter_binding.png" width="800px">

http://git-wip-us.apache.org/repos/asf/zeppelin/blob/998c8f35/zeppelin-interpreter/src/main/java/org/apache/zeppelin/interpreter/Interpreter.java
----------------------------------------------------------------------
diff --git a/zeppelin-interpreter/src/main/java/org/apache/zeppelin/interpreter/Interpreter.java
b/zeppelin-interpreter/src/main/java/org/apache/zeppelin/interpreter/Interpreter.java
index 7114f31..b64530a 100644
--- a/zeppelin-interpreter/src/main/java/org/apache/zeppelin/interpreter/Interpreter.java
+++ b/zeppelin-interpreter/src/main/java/org/apache/zeppelin/interpreter/Interpreter.java
@@ -18,15 +18,19 @@
 package org.apache.zeppelin.interpreter;
 
 
+import java.lang.reflect.Field;
 import java.net.URL;
+import java.util.Arrays;
 import java.util.Collections;
 import java.util.HashMap;
 import java.util.List;
 import java.util.Map;
 import java.util.Properties;
 
-import org.apache.zeppelin.annotation.ZeppelinApi;
+import org.apache.commons.lang.StringUtils;
+import org.apache.commons.lang.reflect.FieldUtils;
 import org.apache.zeppelin.annotation.Experimental;
+import org.apache.zeppelin.annotation.ZeppelinApi;
 import org.apache.zeppelin.interpreter.thrift.InterpreterCompletion;
 import org.apache.zeppelin.scheduler.Scheduler;
 import org.apache.zeppelin.scheduler.SchedulerFactory;
@@ -157,6 +161,8 @@ public abstract class Interpreter {
       }
     }
 
+    replaceContextParameters(p);
+
     return p;
   }
 
@@ -296,6 +302,41 @@ public abstract class Interpreter {
     return null;
   }
 
+  /**
+   * Replace markers #{contextFieldName} by values from {@link InterpreterContext} fields
+   * with same name and marker #{user}. If value == null then replace by empty string.
+   */
+  private void replaceContextParameters(Properties properties) {
+    InterpreterContext interpreterContext = InterpreterContext.get();
+    if (interpreterContext != null) {
+      String markerTemplate = "#\\{%s\\}";
+      List<String> skipFields = Arrays.asList("paragraphTitle", "paragraphId", "paragraphText");
+      List typesToProcess = Arrays.asList(String.class, Double.class, Float.class, Short.class,
+          Byte.class, Character.class, Boolean.class, Integer.class, Long.class);
+      for (String key : properties.stringPropertyNames()) {
+        String p = properties.getProperty(key);
+        if (StringUtils.isNotEmpty(p)) {
+          for (Field field : InterpreterContext.class.getDeclaredFields()) {
+            Class clazz = field.getType();
+            if (!skipFields.contains(field.getName()) && (typesToProcess.contains(clazz)
+                || clazz.isPrimitive())) {
+              Object value = null;
+              try {
+                value = FieldUtils.readField(field, interpreterContext, true);
+              } catch (Exception e) {
+                logger.error("Cannot read value of field {0}", field.getName());
+              }
+              p = p.replaceAll(String.format(markerTemplate, field.getName()),
+                  value != null ? value.toString() : StringUtils.EMPTY);
+            }
+          }
+          p = p.replaceAll(String.format(markerTemplate, "user"),
+              StringUtils.defaultString(userName, StringUtils.EMPTY));
+          properties.setProperty(key, p);
+        }
+      }
+    }
+  }
 
   /**
    * Type of interpreter.

http://git-wip-us.apache.org/repos/asf/zeppelin/blob/998c8f35/zeppelin-interpreter/src/test/java/org/apache/zeppelin/interpreter/InterpreterTest.java
----------------------------------------------------------------------
diff --git a/zeppelin-interpreter/src/test/java/org/apache/zeppelin/interpreter/InterpreterTest.java
b/zeppelin-interpreter/src/test/java/org/apache/zeppelin/interpreter/InterpreterTest.java
index 9eb7932..a9ac1fc 100644
--- a/zeppelin-interpreter/src/test/java/org/apache/zeppelin/interpreter/InterpreterTest.java
+++ b/zeppelin-interpreter/src/test/java/org/apache/zeppelin/interpreter/InterpreterTest.java
@@ -17,13 +17,14 @@
 
 package org.apache.zeppelin.interpreter;
 
-import static org.junit.Assert.assertEquals;
-
 import java.util.Properties;
 
 import org.apache.zeppelin.interpreter.remote.mock.MockInterpreterA;
+import org.apache.zeppelin.user.AuthenticationInfo;
 import org.junit.Test;
 
+import static org.junit.Assert.assertEquals;
+
 public class InterpreterTest {
 
   @Test
@@ -51,4 +52,38 @@ public class InterpreterTest {
     assertEquals("v2", intp.getProperty("p1"));
   }
 
+  @Test
+  public void testPropertyWithReplacedContextFields() {
+    String noteId = "testNoteId";
+    String paragraphTitle = "testParagraphTitle";
+    String paragraphText = "testParagraphText";
+    String paragraphId = "testParagraphId";
+    String user = "username";
+    InterpreterContext.set(new InterpreterContext(noteId,
+        paragraphId,
+        null,
+        paragraphTitle,
+        paragraphText,
+        new AuthenticationInfo("testUser", "testTicket"),
+        null,
+        null,
+        null,
+        null,
+        null,
+        null));
+    Properties p = new Properties();
+    p.put("p1", "replName #{noteId}, #{paragraphTitle}, #{paragraphId}, #{paragraphText},
#{replName}, #{noteId}, #{user}," +
+        " #{authenticationInfo}");
+    MockInterpreterA intp = new MockInterpreterA(p);
+    intp.setUserName(user);
+    String actual = intp.getProperty("p1");
+    InterpreterContext.remove();
+
+    assertEquals(
+        String.format("replName %s, #{paragraphTitle}, #{paragraphId}, #{paragraphText},
, %s, %s, #{authenticationInfo}", noteId,
+            noteId, user),
+        actual
+    );
+  }
+
 }


Mime
View raw message