zeppelin-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From jongy...@apache.org
Subject zeppelin git commit: [zeppelin-3625] [branch-0.8] Pandasql interpreter fails to query over python dataframe
Date Thu, 19 Jul 2018 23:01:23 GMT
Repository: zeppelin
Updated Branches:
  refs/heads/branch-0.8 b682ce16b -> e133507da


[zeppelin-3625] [branch-0.8] Pandasql interpreter fails to query over python dataframe

### What is this PR for?
fix pandasql query over dataframe for `branch-0.8`. the reason in this branch seems to be
different from `master`.  first of all there's NPE with python context being used from pandas
without being initialised. after fixing it there's another problem with `_displayhook`.

### What type of PR is it?
[Bug Fix]

### Todos
* [x] - fix npe
* [x] - fix `_displayhook`
* [x] - test

### What is the Jira issue?
https://issues.apache.org/jira/browse/ZEPPELIN-3625

### How should this be tested?
explained in the issue

### Screenshots (if appropriate)
original NPE different from `master`:
<img width="1232" alt="screen shot 2018-07-15 at 7 26 39 pm" src="https://user-images.githubusercontent.com/1642088/42735603-f01172a0-8891-11e8-87a5-8b359209b889.png">

after fixing npe, issue with `_displayhook`:
<img width="1271" alt="screen shot 2018-07-15 at 7 32 54 pm" src="https://user-images.githubusercontent.com/1642088/42735612-1bc13a34-8892-11e8-9379-af56c46a2ecb.png">
and interpreter logs show
<img width="884" alt="screen shot 2018-07-15 at 7 33 47 pm" src="https://user-images.githubusercontent.com/1642088/42735615-3563e108-8892-11e8-979a-73d9fcf28c0a.png">

at this point not sure why can't execute `__zeppelin__._displayhook()` in ipython kernel and
not sure what's the purpose of this command on pandasql->interpret. removing this command
resolves issue as below. opinions on this are appreciated in case there're better options
<img width="1270" alt="screen shot 2018-07-15 at 7 37 01 pm" src="https://user-images.githubusercontent.com/1642088/42735636-d83db566-8892-11e8-92fc-383957cbcee1.png">

### Questions:
* Does the licenses files need update?
* Is there breaking changes for older versions?
* Does this needs documentation?

Author: Khalid Huseynov <khalidhnv@gmail.com>

Closes #3073 from khalidhuseynov/fix-0.8/ZEPPELIN-3625 and squashes the following commits:

d48418790 [Khalid Huseynov] enable all python tests
bce568971 [Khalid Huseynov] remove _displayhook from pandasql interpret
af11abd1b [Khalid Huseynov] fix npe


Project: http://git-wip-us.apache.org/repos/asf/zeppelin/repo
Commit: http://git-wip-us.apache.org/repos/asf/zeppelin/commit/e133507d
Tree: http://git-wip-us.apache.org/repos/asf/zeppelin/tree/e133507d
Diff: http://git-wip-us.apache.org/repos/asf/zeppelin/diff/e133507d

Branch: refs/heads/branch-0.8
Commit: e133507dac4e16bdb749e822e0d00ab7fba17782
Parents: b682ce1
Author: Khalid Huseynov <khalidhnv@gmail.com>
Authored: Thu Jul 19 12:12:24 2018 +0900
Committer: Jongyoul Lee <jongyoul@apache.org>
Committed: Fri Jul 20 08:01:16 2018 +0900

----------------------------------------------------------------------
 python/pom.xml                                                  | 5 -----
 .../main/java/org/apache/zeppelin/python/PythonInterpreter.java | 2 +-
 .../org/apache/zeppelin/python/PythonInterpreterPandasSql.java  | 2 +-
 3 files changed, 2 insertions(+), 7 deletions(-)
----------------------------------------------------------------------


http://git-wip-us.apache.org/repos/asf/zeppelin/blob/e133507d/python/pom.xml
----------------------------------------------------------------------
diff --git a/python/pom.xml b/python/pom.xml
index 468a7bc..2e6912c 100644
--- a/python/pom.xml
+++ b/python/pom.xml
@@ -35,11 +35,6 @@
   <properties>
     <interpreter.name>python</interpreter.name>
     <python.py4j.version>0.9.2</python.py4j.version>
-    <python.test.exclude>
-        **/PythonInterpreterWithPythonInstalledTest.java,
-        **/PythonInterpreterPandasSqlTest.java,
-        **/PythonInterpreterMatplotlibTest.java
-    </python.test.exclude>
     <pypi.repo.url>https://pypi.python.org/packages</pypi.repo.url>
     <python.py4j.repo.folder>/64/5c/01e13b68e8caafece40d549f232c9b5677ad1016071a48d04cc3895acaa3</python.py4j.repo.folder>
     <grpc.version>1.4.0</grpc.version>

http://git-wip-us.apache.org/repos/asf/zeppelin/blob/e133507d/python/src/main/java/org/apache/zeppelin/python/PythonInterpreter.java
----------------------------------------------------------------------
diff --git a/python/src/main/java/org/apache/zeppelin/python/PythonInterpreter.java b/python/src/main/java/org/apache/zeppelin/python/PythonInterpreter.java
index fab2ed9..cac743d 100644
--- a/python/src/main/java/org/apache/zeppelin/python/PythonInterpreter.java
+++ b/python/src/main/java/org/apache/zeppelin/python/PythonInterpreter.java
@@ -554,7 +554,7 @@ public class PythonInterpreter extends Interpreter implements ExecuteResultHandl
     }
 
     try {
-      interpret(bootstrapCode, context);
+      interpret(bootstrapCode, InterpreterContext.get());
     } catch (InterpreterException e) {
       throw new IOException(e);
     }

http://git-wip-us.apache.org/repos/asf/zeppelin/blob/e133507d/python/src/main/java/org/apache/zeppelin/python/PythonInterpreterPandasSql.java
----------------------------------------------------------------------
diff --git a/python/src/main/java/org/apache/zeppelin/python/PythonInterpreterPandasSql.java
b/python/src/main/java/org/apache/zeppelin/python/PythonInterpreterPandasSql.java
index 54984c3..ea05ce7 100644
--- a/python/src/main/java/org/apache/zeppelin/python/PythonInterpreterPandasSql.java
+++ b/python/src/main/java/org/apache/zeppelin/python/PythonInterpreterPandasSql.java
@@ -90,7 +90,7 @@ public class PythonInterpreterPandasSql extends Interpreter {
     Interpreter python = getPythonInterpreter();
 
     return python.interpret(
-            "__zeppelin__.show(pysqldf('" + st + "'))\n__zeppelin__._displayhook()", context);
+            "__zeppelin__.show(pysqldf('" + st + "'))", context);
   }
 
   @Override


Mime
View raw message