zeppelin-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From zjf...@apache.org
Subject [zeppelin] branch master updated: ZEPPELIN-4062. Don't wait ipython kernel if python process failed
Date Thu, 14 Mar 2019 02:02:29 GMT
This is an automated email from the ASF dual-hosted git repository.

zjffdu pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/zeppelin.git


The following commit(s) were added to refs/heads/master by this push:
     new 65a824e  ZEPPELIN-4062. Don't wait ipython kernel if python process failed
65a824e is described below

commit 65a824e9315649a4d349c7717f6b990d76239c9b
Author: Jeff Zhang <zjffdu@apache.org>
AuthorDate: Wed Mar 13 10:31:00 2019 +0800

    ZEPPELIN-4062. Don't wait ipython kernel if python process failed
    
    ### What is this PR for?
    This PR will exist wait ipython process loop if the python process is failed. This can
save time to exit the waiting loop earlier, otherwise it would wait until timeout (30 seconds
by default)
    
    ### What type of PR is it?
    [Improvement]
    
    ### Todos
    * [ ] - Task
    
    ### What is the Jira issue?
    * https://jira.apache.org/jira/browse/ZEPPELIN-4062
    
    ### How should this be tested?
    * Manually tested.
    
    Before this PR
    ```
     INFO [2019-03-13 10:06:30,070] ({FIFOScheduler-Worker-1} IPythonInterpreter.java[launchIPythonKernel]:313)
- Wait for IPython Kernel to be started
     INFO [2019-03-13 10:06:30,175] ({FIFOScheduler-Worker-1} IPythonInterpreter.java[launchIPythonKernel]:313)
- Wait for IPython Kernel to be started
     INFO [2019-03-13 10:06:30,281] ({FIFOScheduler-Worker-1} IPythonInterpreter.java[launchIPythonKernel]:313)
- Wait for IPython Kernel to be started
     INFO [2019-03-13 10:06:30,386] ({FIFOScheduler-Worker-1} IPythonInterpreter.java[launchIPythonKernel]:313)
- Wait for IPython Kernel to be started
     INFO [2019-03-13 10:06:30,489] ({FIFOScheduler-Worker-1} IPythonInterpreter.java[launchIPythonKernel]:313)
- Wait for IPython Kernel to be started
     INFO [2019-03-13 10:06:30,595] ({FIFOScheduler-Worker-1} IPythonInterpreter.java[launchIPythonKernel]:313)
- Wait for IPython Kernel to be started
     INFO [2019-03-13 10:06:30,700] ({FIFOScheduler-Worker-1} IPythonInterpreter.java[launchIPythonKernel]:313)
- Wait for IPython Kernel to be started
     INFO [2019-03-13 10:06:30,805] ({FIFOScheduler-Worker-1} IPythonInterpreter.java[launchIPythonKernel]:313)
- Wait for IPython Kernel to be started
     WARN [2019-03-13 10:06:30,810] ({Exec Default Executor} IPythonInterpreter.java[onProcessFailed]:419)
- Exception happens in Python Process
    org.apache.commons.exec.ExecuteException: Process exited with an error: 1 (Exit value:
1)
        at org.apache.commons.exec.DefaultExecutor.executeInternal(DefaultExecutor.java:404)
        at org.apache.commons.exec.DefaultExecutor.access$200(DefaultExecutor.java:48)
        at org.apache.commons.exec.DefaultExecutor$1.run(DefaultExecutor.java:200)
        at java.lang.Thread.run(Thread.java:748)
     INFO [2019-03-13 10:06:30,910] ({FIFOScheduler-Worker-1} IPythonInterpreter.java[launchIPythonKernel]:313)
- Wait for IPython Kernel to be started
     INFO [2019-03-13 10:06:31,015] ({FIFOScheduler-Worker-1}
    ```
    
    After this PR
    ```
    INFO [2019-03-13 10:28:12,805] ({FIFOScheduler-Worker-1} IPythonInterpreter.java[launchIPythonKernel]:314)
- Wait for IPython Kernel to be started
     INFO [2019-03-13 10:28:12,907] ({FIFOScheduler-Worker-1} IPythonInterpreter.java[launchIPythonKernel]:314)
- Wait for IPython Kernel to be started
     INFO [2019-03-13 10:28:13,011] ({FIFOScheduler-Worker-1} IPythonInterpreter.java[launchIPythonKernel]:314)
- Wait for IPython Kernel to be started
     INFO [2019-03-13 10:28:13,116] ({FIFOScheduler-Worker-1} IPythonInterpreter.java[launchIPythonKernel]:314)
- Wait for IPython Kernel to be started
     INFO [2019-03-13 10:28:13,220] ({FIFOScheduler-Worker-1} IPythonInterpreter.java[launchIPythonKernel]:314)
- Wait for IPython Kernel to be started
     INFO [2019-03-13 10:28:13,325] ({FIFOScheduler-Worker-1} IPythonInterpreter.java[launchIPythonKernel]:314)
- Wait for IPython Kernel to be started
     WARN [2019-03-13 10:28:13,347] ({Exec Default Executor} IPythonInterpreter.java[onProcessFailed]:423)
- Exception happens in Python Process
    org.apache.commons.exec.ExecuteException: Process exited with an error: 1 (Exit value:
1)
        at org.apache.commons.exec.DefaultExecutor.executeInternal(DefaultExecutor.java:404)
        at org.apache.commons.exec.DefaultExecutor.access$200(DefaultExecutor.java:48)
        at org.apache.commons.exec.DefaultExecutor$1.run(DefaultExecutor.java:200)
        at java.lang.Thread.run(Thread.java:748)
     INFO [2019-03-13 10:28:13,430] ({FIFOScheduler-Worker-1} IPythonInterpreter.java[launchIPythonKernel]:314)
- Wait for IPython Kernel to be started
     WARN [2019-03-13 10:28:13,431] ({FIFOScheduler-Worker-1} PythonInterpreter.java[open]:98)
- Fail to open IPythonInterpreter
    java.lang.RuntimeException: Fail to open IPythonInterpreter
        at org.apache.zeppelin.python.IPythonInterpreter.open(IPythonInterpreter.java:152)
        at org.apache.zeppelin.python.PythonInterpreter.open(PythonInterpreter.java:93)
        at org.apache.zeppelin.interpreter.LazyOpenInterpreter.open(LazyOpenInterpreter.java:69)
        at org.apache.zeppelin.interpreter.remote.RemoteInterpreterServer$InterpretJob.jobRun(RemoteInterpreterServer.java:598)
        at org.apache.zeppelin.interpreter.remote.RemoteInterpreterServer$InterpretJob.jobRun(RemoteInterpreterServer.java:507)
        at org.apache.zeppelin.scheduler.Job.run(Job.java:172)
        at org.apache.zeppelin.scheduler.AbstractScheduler.runJob(AbstractScheduler.java:121)
        at org.apache.zeppelin.scheduler.FIFOScheduler.lambda$runJobInScheduler$0(FIFOScheduler.java:39)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)
    Caused by: java.io.IOException: Fail to launch IPython Kernel as the python process is
failed
        at org.apache.zeppelin.python.IPythonInterpreter.launchIPythonKernel(IPythonInterpreter.java:323)
        at org.apache.zeppelin.python.IPythonInterpreter.open(IPythonInterpreter.java:149)
        ... 10 more
     INFO [2019-03-13 10:28:13,431] ({FIFOScheduler-Worker-1} PythonInterpreter.java[open]:104)
- IPython is not available, use the native PythonInterpreter
    ```
    
    ### Screenshots (if appropriate)
    
    ### Questions:
    * Does the licenses files need update? No
    * Is there breaking changes for older versions? No
    * Does this needs documentation? No
    
    Author: Jeff Zhang <zjffdu@apache.org>
    
    Closes #3332 from zjffdu/ZEPPELIN-4062 and squashes the following commits:
    
    130c2a82a [Jeff Zhang] ZEPPELIN-4062. Don't wait ipython kernel if python process failed
---
 .../main/java/org/apache/zeppelin/python/IPythonInterpreter.java   | 7 ++++++-
 1 file changed, 6 insertions(+), 1 deletion(-)

diff --git a/python/src/main/java/org/apache/zeppelin/python/IPythonInterpreter.java b/python/src/main/java/org/apache/zeppelin/python/IPythonInterpreter.java
index f5c19be..9e23d04 100644
--- a/python/src/main/java/org/apache/zeppelin/python/IPythonInterpreter.java
+++ b/python/src/main/java/org/apache/zeppelin/python/IPythonInterpreter.java
@@ -82,6 +82,7 @@ public class IPythonInterpreter extends Interpreter implements ExecuteResultHand
   private boolean useBuiltinPy4j = true;
   private boolean usePy4JAuth = true;
   private String secret;
+  private volatile boolean pythonProcessFailed = false;
 
   private InterpreterOutputStream interpreterOutput = new InterpreterOutputStream(LOGGER);
 
@@ -293,7 +294,7 @@ public class IPythonInterpreter extends Interpreter implements ExecuteResultHand
 
     // wait until IPython kernel is started or timeout
     long startTime = System.currentTimeMillis();
-    while (true) {
+    while (!pythonProcessFailed) {
       try {
         Thread.sleep(100);
       } catch (InterruptedException e) {
@@ -318,6 +319,9 @@ public class IPythonInterpreter extends Interpreter implements ExecuteResultHand
             + " seconds");
       }
     }
+    if (pythonProcessFailed) {
+      throw new IOException("Fail to launch IPython Kernel as the python process is failed");
+    }
   }
 
   protected Map<String, String> setupIPythonEnv() throws IOException {
@@ -417,6 +421,7 @@ public class IPythonInterpreter extends Interpreter implements ExecuteResultHand
   @Override
   public void onProcessFailed(ExecuteException e) {
     LOGGER.warn("Exception happens in Python Process", e);
+    pythonProcessFailed = true;
   }
 
   static class ProcessLogOutputStream extends LogOutputStream {


Mime
View raw message