zeppelin-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From k..@apache.org
Subject zeppelin git commit: [ZEPPELIN-2582][DOCS] docs for interpreter binding modes
Date Fri, 07 Jul 2017 07:23:57 GMT
Repository: zeppelin
Updated Branches:
  refs/heads/master e1f0a3205 -> 86bc9332e


[ZEPPELIN-2582][DOCS] docs for interpreter binding modes

### What is this PR for?

Updated `interpreter_binding_mode.md` since users are sometimes confused what this mode means
and there is already opened JIRA issue. This documentation will be helpful to Zeppelin users.

**disclaimer: content was copied from [here](https://medium.com/leemoonsoo/apache-zeppelin-interpreter-mode-explained-bae0525d0555)
with author's consent.**

### What type of PR is it?
[Documentation]

### Todos

DONE

### What is the Jira issue?

[ZEPPELIN-2582](https://issues.apache.org/jira/browse/ZEPPELIN-2582)

### How should this be tested?

1. setup rvm 2.1.0+ and install required bundles for `docs/`
2. bundle exec jekyll serve --watch
3. http://localhost:4000/usage/interpreter/interpreter_binding_mode.html

### Screenshots (if appropriate)

![image](https://user-images.githubusercontent.com/4968473/27527441-ac200154-5a86-11e7-8cd8-692aa4fe98f5.png)

![image](https://user-images.githubusercontent.com/4968473/27527446-b3003b24-5a86-11e7-92bb-7b2ae7ae9aa5.png)

![image](https://user-images.githubusercontent.com/4968473/27527449-b802b55c-5a86-11e7-867b-bf86ddc1692a.png)

![image](https://user-images.githubusercontent.com/4968473/27527456-be74afa8-5a86-11e7-9509-861f40ee2175.png)

![image](https://user-images.githubusercontent.com/4968473/27527462-c467abea-5a86-11e7-8d75-0b94962a27b2.png)

![image](https://user-images.githubusercontent.com/4968473/27527464-c6c4ebfa-5a86-11e7-852c-25655ef519dd.png)

### Questions:
* Does the licenses files need update? - NO
* Is there breaking changes for older versions? - NO
* Does this needs documentation? - This PR is about docs.

Author: 1ambda <1amb4a@gmail.com>

Closes #2437 from 1ambda/ZEPPELIN-2582/docs-for-interpreter-modes and squashes the following
commits:

e0bcf26c [1ambda] fix: Docs
a60fdbf2 [1ambda] fix: Correct sentences
cdeb7e8a [1ambda] docs: Update imgs
d60c48ea [1ambda] fix: clarify description
c2317d92 [1ambda] fix: Use basepath
53411350 [1ambda] docs: Remove multiple spark context boxes in scoped mode image
07c4b02a [1ambda] fix: Provide more specific example for per user mode
e385ecf5 [1ambda] docs: Use new images
fc69bc60 [1ambda] docs: Add UI images
56f1a68b [1ambda] docs: Add description for per-user
963f9ce7 [1ambda] docs: Add column to explain sharing objects
33b0736e [1ambda] fix: typo
316c5634 [1ambda] docs: Fix incorrect info for scoped mode, isolated mode
729b56a2 [1ambda] fix: Remove code perspective
af442263 [1ambda] docs: Use lowercase for word note
9ae9552a [1ambda] docs: emphasize per note mode in sopced, iolsated
66d02ba7 [1ambda] docs: Enhance description for ioslated mode
83eb2e5f [1ambda] docs: enhance desc for scoped mode
1eafb647 [1ambda] docs: Add descroptoin for per user, per note
43e8b023 [1ambda] docs: Add summary table of @cacti77
b43a4daf [1ambda] docs: Update interpreter binding mode page
56836205 [1ambda] feat: Add images


Project: http://git-wip-us.apache.org/repos/asf/zeppelin/repo
Commit: http://git-wip-us.apache.org/repos/asf/zeppelin/commit/86bc9332
Tree: http://git-wip-us.apache.org/repos/asf/zeppelin/tree/86bc9332
Diff: http://git-wip-us.apache.org/repos/asf/zeppelin/diff/86bc9332

Branch: refs/heads/master
Commit: 86bc9332e085741d154e9831438d9f4dec920daa
Parents: e1f0a32
Author: 1ambda <1amb4a@gmail.com>
Authored: Wed Jul 5 15:05:49 2017 +0900
Committer: 1ambda <1amb4a@gmail.com>
Committed: Fri Jul 7 16:23:49 2017 +0900

----------------------------------------------------------------------
 ...eter_binding_mode-example-spark-isolated.png | Bin 0 -> 485493 bytes
 ...preter_binding_mode-example-spark-scoped.png | Bin 0 -> 443939 bytes
 ...preter_binding_mode-example-spark-shared.png | Bin 0 -> 212958 bytes
 .../interpreter_binding_mode-isolated.png       | Bin 0 -> 444079 bytes
 .../interpreter_binding_mode-scoped.png         | Bin 0 -> 415883 bytes
 .../interpreter_binding_mode-shared.png         | Bin 0 -> 198508 bytes
 .../interpreter_binding_mode-structure.png      | Bin 0 -> 6459 bytes
 .../interpreter_binding_per_note_user.png       | Bin 0 -> 67866 bytes
 .../interpreter_binding_scoped_isolated.png     | Bin 0 -> 68532 bytes
 .../interpreter/interpreter_binding_mode.md     |  93 +++++++++++++++++++
 10 files changed, 93 insertions(+)
----------------------------------------------------------------------


http://git-wip-us.apache.org/repos/asf/zeppelin/blob/86bc9332/docs/assets/themes/zeppelin/img/docs-img/interpreter_binding_mode-example-spark-isolated.png
----------------------------------------------------------------------
diff --git a/docs/assets/themes/zeppelin/img/docs-img/interpreter_binding_mode-example-spark-isolated.png
b/docs/assets/themes/zeppelin/img/docs-img/interpreter_binding_mode-example-spark-isolated.png
new file mode 100644
index 0000000..90786b6
Binary files /dev/null and b/docs/assets/themes/zeppelin/img/docs-img/interpreter_binding_mode-example-spark-isolated.png
differ

http://git-wip-us.apache.org/repos/asf/zeppelin/blob/86bc9332/docs/assets/themes/zeppelin/img/docs-img/interpreter_binding_mode-example-spark-scoped.png
----------------------------------------------------------------------
diff --git a/docs/assets/themes/zeppelin/img/docs-img/interpreter_binding_mode-example-spark-scoped.png
b/docs/assets/themes/zeppelin/img/docs-img/interpreter_binding_mode-example-spark-scoped.png
new file mode 100644
index 0000000..4377767
Binary files /dev/null and b/docs/assets/themes/zeppelin/img/docs-img/interpreter_binding_mode-example-spark-scoped.png
differ

http://git-wip-us.apache.org/repos/asf/zeppelin/blob/86bc9332/docs/assets/themes/zeppelin/img/docs-img/interpreter_binding_mode-example-spark-shared.png
----------------------------------------------------------------------
diff --git a/docs/assets/themes/zeppelin/img/docs-img/interpreter_binding_mode-example-spark-shared.png
b/docs/assets/themes/zeppelin/img/docs-img/interpreter_binding_mode-example-spark-shared.png
new file mode 100644
index 0000000..49e7649
Binary files /dev/null and b/docs/assets/themes/zeppelin/img/docs-img/interpreter_binding_mode-example-spark-shared.png
differ

http://git-wip-us.apache.org/repos/asf/zeppelin/blob/86bc9332/docs/assets/themes/zeppelin/img/docs-img/interpreter_binding_mode-isolated.png
----------------------------------------------------------------------
diff --git a/docs/assets/themes/zeppelin/img/docs-img/interpreter_binding_mode-isolated.png
b/docs/assets/themes/zeppelin/img/docs-img/interpreter_binding_mode-isolated.png
new file mode 100644
index 0000000..2048362
Binary files /dev/null and b/docs/assets/themes/zeppelin/img/docs-img/interpreter_binding_mode-isolated.png
differ

http://git-wip-us.apache.org/repos/asf/zeppelin/blob/86bc9332/docs/assets/themes/zeppelin/img/docs-img/interpreter_binding_mode-scoped.png
----------------------------------------------------------------------
diff --git a/docs/assets/themes/zeppelin/img/docs-img/interpreter_binding_mode-scoped.png
b/docs/assets/themes/zeppelin/img/docs-img/interpreter_binding_mode-scoped.png
new file mode 100644
index 0000000..b58fa85
Binary files /dev/null and b/docs/assets/themes/zeppelin/img/docs-img/interpreter_binding_mode-scoped.png
differ

http://git-wip-us.apache.org/repos/asf/zeppelin/blob/86bc9332/docs/assets/themes/zeppelin/img/docs-img/interpreter_binding_mode-shared.png
----------------------------------------------------------------------
diff --git a/docs/assets/themes/zeppelin/img/docs-img/interpreter_binding_mode-shared.png
b/docs/assets/themes/zeppelin/img/docs-img/interpreter_binding_mode-shared.png
new file mode 100644
index 0000000..2c1067a
Binary files /dev/null and b/docs/assets/themes/zeppelin/img/docs-img/interpreter_binding_mode-shared.png
differ

http://git-wip-us.apache.org/repos/asf/zeppelin/blob/86bc9332/docs/assets/themes/zeppelin/img/docs-img/interpreter_binding_mode-structure.png
----------------------------------------------------------------------
diff --git a/docs/assets/themes/zeppelin/img/docs-img/interpreter_binding_mode-structure.png
b/docs/assets/themes/zeppelin/img/docs-img/interpreter_binding_mode-structure.png
new file mode 100644
index 0000000..b662f52
Binary files /dev/null and b/docs/assets/themes/zeppelin/img/docs-img/interpreter_binding_mode-structure.png
differ

http://git-wip-us.apache.org/repos/asf/zeppelin/blob/86bc9332/docs/assets/themes/zeppelin/img/docs-img/interpreter_binding_per_note_user.png
----------------------------------------------------------------------
diff --git a/docs/assets/themes/zeppelin/img/docs-img/interpreter_binding_per_note_user.png
b/docs/assets/themes/zeppelin/img/docs-img/interpreter_binding_per_note_user.png
new file mode 100644
index 0000000..23f375a
Binary files /dev/null and b/docs/assets/themes/zeppelin/img/docs-img/interpreter_binding_per_note_user.png
differ

http://git-wip-us.apache.org/repos/asf/zeppelin/blob/86bc9332/docs/assets/themes/zeppelin/img/docs-img/interpreter_binding_scoped_isolated.png
----------------------------------------------------------------------
diff --git a/docs/assets/themes/zeppelin/img/docs-img/interpreter_binding_scoped_isolated.png
b/docs/assets/themes/zeppelin/img/docs-img/interpreter_binding_scoped_isolated.png
new file mode 100644
index 0000000..abcac0e
Binary files /dev/null and b/docs/assets/themes/zeppelin/img/docs-img/interpreter_binding_scoped_isolated.png
differ

http://git-wip-us.apache.org/repos/asf/zeppelin/blob/86bc9332/docs/usage/interpreter/interpreter_binding_mode.md
----------------------------------------------------------------------
diff --git a/docs/usage/interpreter/interpreter_binding_mode.md b/docs/usage/interpreter/interpreter_binding_mode.md
index 0981ad1..55c819c 100644
--- a/docs/usage/interpreter/interpreter_binding_mode.md
+++ b/docs/usage/interpreter/interpreter_binding_mode.md
@@ -25,4 +25,97 @@ limitations under the License.
 
 ## Overview
 
+<center><img src="{{BASE_PATH}}/assets/themes/zeppelin/img/docs-img/interpreter_binding_per_note_user.png"
height="70%" width="70%"></center>
+<center><img src="{{BASE_PATH}}/assets/themes/zeppelin/img/docs-img/interpreter_binding_scoped_isolated.png"
height="70%" width="70%"></center>
 
+<br/>
+
+Interpreter Process is a JVM process that communicates with Zeppelin daemon using thrift.

+Each interpreter process has a single interpreter group, and this interpreter group can have
one or more instances of an interpreter.
+(See [here](../../development/writing_zeppelin_interpreter.html) to understand more about
its internal structure.) 
+
+Zeppelin provides 3 different modes to run interpreter process: **shared**, **scoped** and
**isolated**.   
+Also, the user can specify the scope of these modes: **per** user or **per note**.
+These 3 modes give flexibility to fit Zeppelin into any type of use cases.
+
+In this documentation, we mainly discuss the **per note** scope in combination with the **shared**,
**scoped** and **isolated** modes.
+
+## Shared Mode
+
+<div class="text-center">
+    <img src="{{BASE_PATH}}/assets/themes/zeppelin/img/docs-img/interpreter_binding_mode-shared.png"
height="40%" width="40%">
+</div>
+<br/>
+
+In **Shared** mode, single JVM process and a single session serves all notes. As a result,
`note A` can access variables (e.g python, scala, ..) directly created from other notes..

+
+## Scoped Mode
+
+<div class="text-center">
+    <img src="{{BASE_PATH}}/assets/themes/zeppelin/img/docs-img/interpreter_binding_mode-scoped.png">
+</div>
+<br/>
+
+In **Scoped** mode, Zeppelin still runs a single interpreter JVM process but, in the case
of per note scope, each note runs in its own dedicated session. 
+(Note it is still possible to share objects between these notes via [ResourcePool](../../interpreter/spark.html#object-exchange))

+
+## Isolated Mode
+
+<div class="text-center">
+    <img src="{{BASE_PATH}}/assets/themes/zeppelin/img/docs-img/interpreter_binding_mode-isolated.png">
+</div>
+<br/>
+
+**Isolated** mode runs a separate interpreter process for each note in the case of **per
note** scope. 
+So, each note has an absolutely isolated session. (But it is still possible to share objects
via [ResourcePool](../../interpreter/spark.html#object-exchange)) 
+
+## Which mode should I use?
+
+<br/>
+
+Mode | Each notebook...	| Benefits | Disadvantages | Sharing objects
+--- | --- | --- | --- | ---
+**shared** |  Shares a single session in a single interpreter process (JVM) |  Low resource
utilization and it's easy to share data between notebooks |  All notebooks are affected if
the interpreter process dies | Can share directly
+**scoped** | Has its own session in the same interpreter process (JVM) | Less resource utilization
than isolated mode |  All notebooks are affected if the interpreter process dies | Can't share
directly, but it's possible to share objects via [ResourcePool](../../interpreter/spark.html#object-exchange))

+**isolated** | Has its own Interpreter Process | One notebook is not affected directly by
other notebooks (**per note**) | Can't share data between notebooks easily (**per note**)
| Can't share directly, but it's possible to share objects via [ResourcePool](../../interpreter/spark.html#object-exchange))

+
+In the case of the **per user** scope (available in a multi-user environment), Zeppelin manages
interpreter sessions on a per user basis rather than a per note basis. For example:
+ 
+- In **scoped + per user** mode, `User A`'s notes **might** be affected by `User B`'s notes.
(e.g JVM dies, ...) Because all notes are running on the same JVM
+- On the other hand, **isolated + per user** mode, `User A`'s notes will not be affected
by others' notes which running on separated JVMs
+
+<br/>
+
+Each Interpreter implementation may have different characteristics depending on the back
end system that they integrate. And 3 interpreter modes can be used differently.
+Let’s take a look how Spark Interpreter implementation uses these 3 interpreter modes with
**per note** scope, as an example.
+Spark Interpreter implementation includes 4 different interpreters in the group: Spark, SparkSQL,
Pyspark and SparkR. 
+SparkInterpreter instance embeds Scala REPL for interactive Spark API execution.
+
+<br/>
+
+<div class="text-center">
+    <img src="{{BASE_PATH}}/assets/themes/zeppelin/img/docs-img/interpreter_binding_mode-example-spark-shared.png"
height="40%" width="40%">
+</div>
+<br/>
+
+In Shared mode, a SparkContext and a Scala REPL is being shared among all interpreters in
the group. 
+So every note will be sharing single SparkContext and single Scala REPL. 
+In this mode, if `Note A` defines variable ‘a’ then `Note B` not only able to read variable
‘a’ but also able to override the variable.
+
+<div class="text-center">
+    <img src="{{BASE_PATH}}/assets/themes/zeppelin/img/docs-img/interpreter_binding_mode-example-spark-scoped.png">
+</div>
+<br/>
+
+In Scoped mode, each note has its own Scala REPL. 
+So variable defined in a note can not be read or overridden in another note. 
+However, a single SparkContext still serves all the sessions.
+And all the jobs are submitted to this SparkContext and the fair scheduler schedules the
jobs.
+This could be useful when user does not want to share Scala session, but want to keep single
Spark application and leverage its fair scheduler.
+
+In Isolated mode, each note has its own SparkContext and Scala REPL.
+
+<div class="text-center">
+    <img src="{{BASE_PATH}}/assets/themes/zeppelin/img/docs-img/interpreter_binding_mode-example-spark-isolated.png">
+</div>
+<br/>


Mime
View raw message