zeppelin-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From m...@apache.org
Subject svn commit: r1768714 - in /zeppelin/site/docs/0.7.0-SNAPSHOT: assets/themes/zeppelin/img/docs-img/matplotlibAngularExample.gif atom.xml interpreter/python.html interpreter/spark.html rss.xml search_data.json security/shiroauthentication.html
Date Tue, 08 Nov 2016 15:33:30 GMT
Author: moon
Date: Tue Nov  8 15:33:30 2016
New Revision: 1768714

URL: http://svn.apache.org/viewvc?rev=1768714&view=rev
Log:
Update doc for ZEPPELIN-1611, ZEPPELIN-1345

Added:
    zeppelin/site/docs/0.7.0-SNAPSHOT/assets/themes/zeppelin/img/docs-img/matplotlibAngularExample.gif   (with props)
Modified:
    zeppelin/site/docs/0.7.0-SNAPSHOT/atom.xml
    zeppelin/site/docs/0.7.0-SNAPSHOT/interpreter/python.html
    zeppelin/site/docs/0.7.0-SNAPSHOT/interpreter/spark.html
    zeppelin/site/docs/0.7.0-SNAPSHOT/rss.xml
    zeppelin/site/docs/0.7.0-SNAPSHOT/search_data.json
    zeppelin/site/docs/0.7.0-SNAPSHOT/security/shiroauthentication.html

Added: zeppelin/site/docs/0.7.0-SNAPSHOT/assets/themes/zeppelin/img/docs-img/matplotlibAngularExample.gif
URL: http://svn.apache.org/viewvc/zeppelin/site/docs/0.7.0-SNAPSHOT/assets/themes/zeppelin/img/docs-img/matplotlibAngularExample.gif?rev=1768714&view=auto
==============================================================================
Binary file - no diff available.

Propchange: zeppelin/site/docs/0.7.0-SNAPSHOT/assets/themes/zeppelin/img/docs-img/matplotlibAngularExample.gif
------------------------------------------------------------------------------
    svn:mime-type = application/octet-stream

Modified: zeppelin/site/docs/0.7.0-SNAPSHOT/atom.xml
URL: http://svn.apache.org/viewvc/zeppelin/site/docs/0.7.0-SNAPSHOT/atom.xml?rev=1768714&r1=1768713&r2=1768714&view=diff
==============================================================================
--- zeppelin/site/docs/0.7.0-SNAPSHOT/atom.xml (original)
+++ zeppelin/site/docs/0.7.0-SNAPSHOT/atom.xml Tue Nov  8 15:33:30 2016
@@ -4,7 +4,7 @@
  <title>Apache Zeppelin</title>
  <link href="http://zeppelin.apache.org/" rel="self"/>
  <link href="http://zeppelin.apache.org"/>
- <updated>2016-11-05T13:46:01+09:00</updated>
+ <updated>2016-11-08T07:32:29-08:00</updated>
  <id>http://zeppelin.apache.org</id>
  <author>
    <name>The Apache Software Foundation</name>

Modified: zeppelin/site/docs/0.7.0-SNAPSHOT/interpreter/python.html
URL: http://svn.apache.org/viewvc/zeppelin/site/docs/0.7.0-SNAPSHOT/interpreter/python.html?rev=1768714&r1=1768713&r2=1768714&view=diff
==============================================================================
--- zeppelin/site/docs/0.7.0-SNAPSHOT/interpreter/python.html (original)
+++ zeppelin/site/docs/0.7.0-SNAPSHOT/interpreter/python.html Tue Nov  8 15:33:30 2016
@@ -270,16 +270,29 @@ limitations under the License.
 
 <h2>Matplotlib integration</h2>
 
-<p>The python interpreter can display matplotlib graph with the function <code>z.show()</code>.
- You need to have matplotlib module installed and a XServer running to use this functionality!</p>
+<p>The python interpreter can display matplotlib figures inline automatically using the <code>pyplot</code> module:</p>
 <div class="highlight"><pre><code class="python language-python" data-lang="python"><span class="o">%</span><span class="n">python</span>
 <span class="kn">import</span> <span class="nn">matplotlib.pyplot</span> <span class="kn">as</span> <span class="nn">plt</span>
-<span class="n">plt</span><span class="o">.</span><span class="n">figure</span><span class="p">()</span>
-<span class="p">(</span><span class="o">..</span> <span class="o">..</span><span class="p">)</span>
-<span class="n">z</span><span class="o">.</span><span class="n">show</span><span class="p">(</span><span class="n">plt</span><span class="p">)</span>
-<span class="n">plt</span><span class="o">.</span><span class="n">close</span><span class="p">()</span>
+<span class="n">plt</span><span class="o">.</span><span class="n">plot</span><span class="p">([</span><span class="mi">1</span><span class="p">,</span> <span class="mi">2</span><span class="p">,</span> <span class="mi">3</span><span class="p">])</span>
 </code></pre></div>
-<p>The <code>z.show()</code> function can take optional parameters to adapt graph dimensions (width and height) as well as output format (png or optionally svg).</p>
+<p>This is the recommended method for using matplotlib from within a Zeppelin notebook. The output of this command will by default be converted to HTML by implicitly making use of the <code>%html</code> magic. Additional configuration can be achieved using the builtin <code>z.configure_mpl()</code> method. For example, </p>
+<div class="highlight"><pre><code class="python language-python" data-lang="python"><span class="n">z</span><span class="o">.</span><span class="n">configure_mpl</span><span class="p">(</span><span class="n">width</span><span class="o">=</span><span class="mi">400</span><span class="p">,</span> <span class="n">height</span><span class="o">=</span><span class="mi">300</span><span class="p">,</span> <span class="n">fmt</span><span class="o">=</span><span class="s">&#39;svg&#39;</span><span class="p">)</span>
+<span class="n">plt</span><span class="o">.</span><span class="n">plot</span><span class="p">([</span><span class="mi">1</span><span class="p">,</span> <span class="mi">2</span><span class="p">,</span> <span class="mi">3</span><span class="p">])</span>
+</code></pre></div>
+<p>Will produce a 400x300 image in SVG format, which by default are normally 600x400 and PNG respectively. In the future, another option called <code>angular</code> can be used to make it possible to update a plot produced from one paragraph directly from another (the output will be <code>%angular</code> instead of <code>%html</code>). However, this feature is already available in the <code>pyspark</code> interpreter. More details can be found in the included &quot;Zeppelin Tutorial: Python - matplotlib basic&quot; tutorial notebook. </p>
+
+<p>If Zeppelin cannot find the matplotlib backend files (which should usually be found in <code>$ZEPPELIN_HOME/interpreter/lib/python</code>) in your <code>PYTHONPATH</code>, then the backend will automatically be set to agg, and the (otherwise deprecated) instructions below can be used for more limited inline plotting.</p>
+
+<p>If you are unable to load the inline backend, use <code>z.show(plt)</code>:
+ <code>python
+%python
+import matplotlib.pyplot as plt
+plt.figure()
+(.. ..)
+z.show(plt)
+plt.close()
+</code>
+The <code>z.show()</code> function can take optional parameters to adapt graph dimensions (width and height) as well as output format (png or optionally svg).</p>
 <div class="highlight"><pre><code class="python language-python" data-lang="python"><span class="o">%</span><span class="n">python</span>
 <span class="n">z</span><span class="o">.</span><span class="n">show</span><span class="p">(</span><span class="n">plt</span><span class="p">,</span> <span class="n">width</span><span class="o">=</span><span class="s">&#39;50px&#39;</span><span class="p">)</span>
 <span class="n">z</span><span class="o">.</span><span class="n">show</span><span class="p">(</span><span class="n">plt</span><span class="p">,</span> <span class="n">height</span><span class="o">=</span><span class="s">&#39;150px&#39;</span><span class="p">,</span> <span class="n">fmt</span><span class="o">=</span><span class="s">&#39;svg&#39;</span><span class="p">)</span>

Modified: zeppelin/site/docs/0.7.0-SNAPSHOT/interpreter/spark.html
URL: http://svn.apache.org/viewvc/zeppelin/site/docs/0.7.0-SNAPSHOT/interpreter/spark.html?rev=1768714&r1=1768713&r2=1768714&view=diff
==============================================================================
--- zeppelin/site/docs/0.7.0-SNAPSHOT/interpreter/spark.html (original)
+++ zeppelin/site/docs/0.7.0-SNAPSHOT/interpreter/spark.html Tue Nov  8 15:33:30 2016
@@ -553,6 +553,12 @@ In Scala and Python environments, you ca
 </code></pre></div>
 <p>To learn more about dynamic form, checkout <a href="../manual/dynamicform.html">Dynamic Form</a>.</p>
 
+<h2>Matplotlib Integration (pyspark)</h2>
+
+<p>Both the <code>python</code> and <code>pyspark</code> interpreters have built-in support for inline visualization using <code>matplotlib</code>, a popular plotting library for python. More details can be found in the <a href="../interpreter/python.html">python interpreter documentation</a>, since matplotlib support is identical. More advanced interactive plotting can be done with pyspark through utilizing Zeppelin&#39;s built-in <a href="../displaysystem/back-end-angular.html">Angular Display System</a>, as shown below:</p>
+
+<p><img class="img-responsive" src="../assets/themes/zeppelin/img/docs-img/matplotlibAngularExample.gif" /></p>
+
 <h2>Interpreter setting option</h2>
 
 <p>You can choose one of <code>shared</code>, <code>scoped</code> and <code>isolated</code> options wheh you configure Spark interpreter. Spark interpreter creates separated Scala compiler per each notebook but share a single SparkContext in <code>scoped</code> mode (experimental). It creates separated SparkContext per each notebook in <code>isolated</code> mode.</p>

Modified: zeppelin/site/docs/0.7.0-SNAPSHOT/rss.xml
URL: http://svn.apache.org/viewvc/zeppelin/site/docs/0.7.0-SNAPSHOT/rss.xml?rev=1768714&r1=1768713&r2=1768714&view=diff
==============================================================================
--- zeppelin/site/docs/0.7.0-SNAPSHOT/rss.xml (original)
+++ zeppelin/site/docs/0.7.0-SNAPSHOT/rss.xml Tue Nov  8 15:33:30 2016
@@ -5,8 +5,8 @@
         <description>Apache Zeppelin - The Apache Software Foundation</description>
         <link>http://zeppelin.apache.org</link>
         <link>http://zeppelin.apache.org</link>
-        <lastBuildDate>2016-11-05T13:46:01+09:00</lastBuildDate>
-        <pubDate>2016-11-05T13:46:01+09:00</pubDate>
+        <lastBuildDate>2016-11-08T07:32:29-08:00</lastBuildDate>
+        <pubDate>2016-11-08T07:32:29-08:00</pubDate>
         <ttl>1800</ttl>
 
 

Modified: zeppelin/site/docs/0.7.0-SNAPSHOT/search_data.json
URL: http://svn.apache.org/viewvc/zeppelin/site/docs/0.7.0-SNAPSHOT/search_data.json?rev=1768714&r1=1768713&r2=1768714&view=diff
==============================================================================
--- zeppelin/site/docs/0.7.0-SNAPSHOT/search_data.json (original)
+++ zeppelin/site/docs/0.7.0-SNAPSHOT/search_data.json Tue Nov  8 15:33:30 2016
@@ -336,7 +336,7 @@
 
     "/interpreter/python.html": {
       "title": "Python 2 &amp; 3 Interpreter for Apache Zeppelin",
-      "content"  : "&lt;!--Licensed under the Apache License, Version 2.0 (the &quot;License&quot;);you may not use this file except in compliance with the License.You may obtain a copy of the License athttp://www.apache.org/licenses/LICENSE-2.0Unless required by applicable law or agreed to in writing, softwaredistributed under the License is distributed on an &quot;AS IS&quot; BASIS,WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.See the License for the specific language governing permissions andlimitations under the License.--&gt;Python 2 &amp;amp; 3 Interpreter for Apache ZeppelinConfiguration      Property    Default    Description        zeppelin.python    python    Path of the already installed Python binary (could be python2 or python3).    If python is not in your $PATH you can set the absolute directory (example : /usr/bin/python)            zeppelin.python.maxResult    1000    Max number of dataframe rows to display.  Enabling Python InterpreterIn a
  notebook, to enable the Python interpreter, click on the Gear icon and select PythonUsing the Python InterpreterIn a paragraph, use %python to select the Python interpreter and then input all commands.The interpreter can only work if you already have python installed (the interpreter doesn&amp;#39;t bring it own python binaries).To access the help, type help()Python modulesThe interpreter can use all modules already installed (with pip, easy_install...)Using Zeppelin Dynamic FormsYou can leverage Zeppelin Dynamic Form inside your Python code.Zeppelin Dynamic Form can only be used if py4j Python library is installed in your system. If not, you can install it with pip install py4j.Example : %python### Input formprint (z.input(&amp;quot;f1&amp;quot;,&amp;quot;defaultValue&amp;quot;))### Select formprint (z.select(&amp;quot;f1&amp;quot;,[(&amp;quot;o1&amp;quot;,&amp;quot;1&amp;quot;),(&amp;quot;o2&amp;quot;,&amp;quot;2&amp;quot;)],&amp;quot;2&amp;quot;))### Checkbox formprint(&amp;quot
 ;&amp;quot;.join(z.checkbox(&amp;quot;f3&amp;quot;, [(&amp;quot;o1&amp;quot;,&amp;quot;1&amp;quot;), (&amp;quot;o2&amp;quot;,&amp;quot;2&amp;quot;)],[&amp;quot;1&amp;quot;])))Zeppelin features not fully supported by the Python InterpreterInterrupt a paragraph execution (cancel() method) is currently only supported in Linux and MacOs. If interpreter runs in another operating system (for instance MS Windows) , interrupt a paragraph will close the whole interpreter. A JIRA ticket (ZEPPELIN-893) is opened to implement this feature in a next release of the interpreter.Progression bar in webUI  (getProgress() method) is currently not implemented.Code-completion is currently not implemented.Matplotlib integrationThe python interpreter can display matplotlib graph with the function z.show(). You need to have matplotlib module installed and a XServer running to use this functionality!%pythonimport matplotlib.pyplot as pltplt.figure()(.. ..)z.show(plt)plt.close()The z.show() function can take
  optional parameters to adapt graph dimensions (width and height) as well as output format (png or optionally svg).%pythonz.show(plt, width=&amp;#39;50px&amp;#39;)z.show(plt, height=&amp;#39;150px&amp;#39;, fmt=&amp;#39;svg&amp;#39;)Pandas integrationApache Zeppelin Table Display System provides built-in data visualization capabilities. Python interpreter leverages it to visualize Pandas DataFrames though similar z.show() API, same as with Matplotlib integration.Example:import pandas as pdrates = pd.read_csv(&amp;quot;bank.csv&amp;quot;, sep=&amp;quot;;&amp;quot;)z.show(rates)SQL over Pandas DataFramesThere is a convenience %python.sql interpreter that matches Apache Spark experience in Zeppelin and enables usage of SQL language to query Pandas DataFrames and visualization of results though built-in Table Display System.Pre-requestsPandas pip install pandasPandaSQL pip install -U pandasqlIn case default binded interpreter is Python (first in the interpreter list, under the Gear Icon
 ), you can just use it as %sql i.efirst paragraphimport pandas as pdrates = pd.read_csv(&amp;quot;bank.csv&amp;quot;, sep=&amp;quot;;&amp;quot;)next paragraph%sqlSELECT * FROM rates WHERE age &amp;lt; 40Otherwise it can be referred to as %python.sqlTechnical descriptionFor in-depth technical details on current implementation please refer to python/README.md.",
+      "content"  : "&lt;!--Licensed under the Apache License, Version 2.0 (the &quot;License&quot;);you may not use this file except in compliance with the License.You may obtain a copy of the License athttp://www.apache.org/licenses/LICENSE-2.0Unless required by applicable law or agreed to in writing, softwaredistributed under the License is distributed on an &quot;AS IS&quot; BASIS,WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.See the License for the specific language governing permissions andlimitations under the License.--&gt;Python 2 &amp;amp; 3 Interpreter for Apache ZeppelinConfiguration      Property    Default    Description        zeppelin.python    python    Path of the already installed Python binary (could be python2 or python3).    If python is not in your $PATH you can set the absolute directory (example : /usr/bin/python)            zeppelin.python.maxResult    1000    Max number of dataframe rows to display.  Enabling Python InterpreterIn a
  notebook, to enable the Python interpreter, click on the Gear icon and select PythonUsing the Python InterpreterIn a paragraph, use %python to select the Python interpreter and then input all commands.The interpreter can only work if you already have python installed (the interpreter doesn&amp;#39;t bring it own python binaries).To access the help, type help()Python modulesThe interpreter can use all modules already installed (with pip, easy_install...)Using Zeppelin Dynamic FormsYou can leverage Zeppelin Dynamic Form inside your Python code.Zeppelin Dynamic Form can only be used if py4j Python library is installed in your system. If not, you can install it with pip install py4j.Example : %python### Input formprint (z.input(&amp;quot;f1&amp;quot;,&amp;quot;defaultValue&amp;quot;))### Select formprint (z.select(&amp;quot;f1&amp;quot;,[(&amp;quot;o1&amp;quot;,&amp;quot;1&amp;quot;),(&amp;quot;o2&amp;quot;,&amp;quot;2&amp;quot;)],&amp;quot;2&amp;quot;))### Checkbox formprint(&amp;quot
 ;&amp;quot;.join(z.checkbox(&amp;quot;f3&amp;quot;, [(&amp;quot;o1&amp;quot;,&amp;quot;1&amp;quot;), (&amp;quot;o2&amp;quot;,&amp;quot;2&amp;quot;)],[&amp;quot;1&amp;quot;])))Zeppelin features not fully supported by the Python InterpreterInterrupt a paragraph execution (cancel() method) is currently only supported in Linux and MacOs. If interpreter runs in another operating system (for instance MS Windows) , interrupt a paragraph will close the whole interpreter. A JIRA ticket (ZEPPELIN-893) is opened to implement this feature in a next release of the interpreter.Progression bar in webUI  (getProgress() method) is currently not implemented.Code-completion is currently not implemented.Matplotlib integrationThe python interpreter can display matplotlib figures inline automatically using the pyplot module:%pythonimport matplotlib.pyplot as pltplt.plot([1, 2, 3])This is the recommended method for using matplotlib from within a Zeppelin notebook. The output of this command will by defaul
 t be converted to HTML by implicitly making use of the %html magic. Additional configuration can be achieved using the builtin z.configure_mpl() method. For example, z.configure_mpl(width=400, height=300, fmt=&amp;#39;svg&amp;#39;)plt.plot([1, 2, 3])Will produce a 400x300 image in SVG format, which by default are normally 600x400 and PNG respectively. In the future, another option called angular can be used to make it possible to update a plot produced from one paragraph directly from another (the output will be %angular instead of %html). However, this feature is already available in the pyspark interpreter. More details can be found in the included &amp;quot;Zeppelin Tutorial: Python - matplotlib basic&amp;quot; tutorial notebook. If Zeppelin cannot find the matplotlib backend files (which should usually be found in $ZEPPELIN_HOME/interpreter/lib/python) in your PYTHONPATH, then the backend will automatically be set to agg, and the (otherwise deprecated) instructions below can be 
 used for more limited inline plotting.If you are unable to load the inline backend, use z.show(plt): python%pythonimport matplotlib.pyplot as pltplt.figure()(.. ..)z.show(plt)plt.close()The z.show() function can take optional parameters to adapt graph dimensions (width and height) as well as output format (png or optionally svg).%pythonz.show(plt, width=&amp;#39;50px&amp;#39;)z.show(plt, height=&amp;#39;150px&amp;#39;, fmt=&amp;#39;svg&amp;#39;)Pandas integrationApache Zeppelin Table Display System provides built-in data visualization capabilities. Python interpreter leverages it to visualize Pandas DataFrames though similar z.show() API, same as with Matplotlib integration.Example:import pandas as pdrates = pd.read_csv(&amp;quot;bank.csv&amp;quot;, sep=&amp;quot;;&amp;quot;)z.show(rates)SQL over Pandas DataFramesThere is a convenience %python.sql interpreter that matches Apache Spark experience in Zeppelin and enables usage of SQL language to query Pandas DataFrames and visualizati
 on of results though built-in Table Display System.Pre-requestsPandas pip install pandasPandaSQL pip install -U pandasqlIn case default binded interpreter is Python (first in the interpreter list, under the Gear Icon), you can just use it as %sql i.efirst paragraphimport pandas as pdrates = pd.read_csv(&amp;quot;bank.csv&amp;quot;, sep=&amp;quot;;&amp;quot;)next paragraph%sqlSELECT * FROM rates WHERE age &amp;lt; 40Otherwise it can be referred to as %python.sqlTechnical descriptionFor in-depth technical details on current implementation please refer to python/README.md.",
       "url": " /interpreter/python.html",
       "group": "interpreter",
       "excerpt": "Python is a programming language that lets you work quickly and integrate systems more effectively."
@@ -380,7 +380,7 @@
 
     "/interpreter/spark.html": {
       "title": "Apache Spark Interpreter for Apache Zeppelin",
-      "content"  : "&lt;!--Licensed under the Apache License, Version 2.0 (the &quot;License&quot;);you may not use this file except in compliance with the License.You may obtain a copy of the License athttp://www.apache.org/licenses/LICENSE-2.0Unless required by applicable law or agreed to in writing, softwaredistributed under the License is distributed on an &quot;AS IS&quot; BASIS,WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.See the License for the specific language governing permissions andlimitations under the License.--&gt;Spark Interpreter for Apache ZeppelinOverviewApache Spark is a fast and general-purpose cluster computing system.It provides high-level APIs in Java, Scala, Python and R, and an optimized engine that supports general execution graphs.Apache Spark is supported in Zeppelin with Spark interpreter group which consists of below five interpreters.      Name    Class    Description        %spark    SparkInterpreter    Creates a SparkConte
 xt and provides a Scala environment        %spark.pyspark    PySparkInterpreter    Provides a Python environment        %spark.r    SparkRInterpreter    Provides an R environment with SparkR support        %spark.sql    SparkSQLInterpreter    Provides a SQL environment        %spark.dep    DepInterpreter    Dependency loader  ConfigurationThe Spark interpreter can be configured with properties provided by Zeppelin.You can also set other Spark properties which are not listed in the table. For a list of additional properties, refer to Spark Available Properties.      Property    Default    Description        args        Spark commandline args      master    local[*]    Spark master uri.  ex) spark://masterhost:7077      spark.app.name    Zeppelin    The name of spark application.        spark.cores.max        Total number of cores to use.  Empty value uses all available core.        spark.executor.memory     1g    Executor memory per worker instance.  ex) 512m, 32g        zeppelin.dep
 .additionalRemoteRepository    spark-packages,  http://dl.bintray.com/spark-packages/maven,  false;    A list of id,remote-repository-URL,is-snapshot;  for each remote repository.        zeppelin.dep.localrepo    local-repo    Local repository for dependency loader        zeppelin.pyspark.python    python    Python command to run pyspark with        zeppelin.spark.concurrentSQL    false    Execute multiple SQL concurrently if set true.        zeppelin.spark.maxResult    1000    Max number of Spark SQL result to display.        zeppelin.spark.printREPLOutput    true    Print REPL output        zeppelin.spark.useHiveContext    true    Use HiveContext instead of SQLContext if it is true.        zeppelin.spark.importImplicit    true    Import implicits, UDF collection, and sql if set true.  Without any configuration, Spark interpreter works out of box in local mode. But if you want to connect to your Spark cluster, you&amp;#39;ll need to follow below two simple steps.1. Export SPARK_HOM
 EIn conf/zeppelin-env.sh, export SPARK_HOME environment variable with your Spark installation path.For example,export SPARK_HOME=/usr/lib/sparkYou can optionally export HADOOP_CONF_DIR and SPARK_SUBMIT_OPTIONSexport HADOOP_CONF_DIR=/usr/lib/hadoopexport SPARK_SUBMIT_OPTIONS=&amp;quot;--packages com.databricks:spark-csv_2.10:1.2.0&amp;quot;For Windows, ensure you have winutils.exe in %HADOOP_HOME%bin. Please see Problems running Hadoop on Windows for the details.2. Set master in Interpreter menuAfter start Zeppelin, go to Interpreter menu and edit master property in your Spark interpreter setting. The value may vary depending on your Spark cluster deployment type.For example,local[*] in local modespark://master:7077 in standalone clusteryarn-client in Yarn client modemesos://host:5050 in Mesos clusterThat&amp;#39;s it. Zeppelin will work with any version of Spark and any deployment type without rebuilding Zeppelin in this way. For the further information about Spark &amp;amp; Zeppeli
 n version compatibility, please refer to &amp;quot;Available Interpreters&amp;quot; section in Zeppelin download page.Note that without exporting SPARK_HOME, it&amp;#39;s running in local mode with included version of Spark. The included version may vary depending on the build profile.SparkContext, SQLContext, SparkSession, ZeppelinContextSparkContext, SQLContext and ZeppelinContext are automatically created and exposed as variable names sc, sqlContext and z, respectively, in Scala, Python and R environments.Staring from 0.6.1 SparkSession is available as variable spark when you are using Spark 2.x.Note that Scala/Python/R environment shares the same SparkContext, SQLContext and ZeppelinContext instance. Dependency ManagementThere are two ways to load external libraries in Spark interpreter. First is using interpreter setting menu and second is loading Spark properties.1. Setting Dependencies via Interpreter SettingPlease see Dependency Management for the details.2. Loading Spark Pr
 opertiesOnce SPARK_HOME is set in conf/zeppelin-env.sh, Zeppelin uses spark-submit as spark interpreter runner. spark-submit supports two ways to load configurations. The first is command line options such as --master and Zeppelin can pass these options to spark-submit by exporting SPARK_SUBMIT_OPTIONS in conf/zeppelin-env.sh. Second is reading configuration options from SPARK_HOME/conf/spark-defaults.conf. Spark properties that user can set to distribute libraries are:      spark-defaults.conf    SPARK_SUBMIT_OPTIONS    Description        spark.jars    --jars    Comma-separated list of local jars to include on the driver and executor classpaths.        spark.jars.packages    --packages    Comma-separated list of maven coordinates of jars to include on the driver and executor classpaths. Will search the local maven repo, then maven central and any additional remote repositories given by --repositories. The format for the coordinates should be groupId:artifactId:version.        spark
 .files    --files    Comma-separated list of files to be placed in the working directory of each executor.  Here are few examples:SPARK_SUBMIT_OPTIONS in conf/zeppelin-env.shexport SPARK_SUBMIT_OPTIONS=&amp;quot;--packages com.databricks:spark-csv_2.10:1.2.0 --jars /path/mylib1.jar,/path/mylib2.jar --files /path/mylib1.py,/path/mylib2.zip,/path/mylib3.egg&amp;quot;SPARK_HOME/conf/spark-defaults.confspark.jars        /path/mylib1.jar,/path/mylib2.jarspark.jars.packages   com.databricks:spark-csv_2.10:1.2.0spark.files       /path/mylib1.py,/path/mylib2.egg,/path/mylib3.zip3. Dynamic Dependency Loading via %spark.dep interpreterNote: %spark.dep interpreter is deprecated since v0.6.0.%spark.dep interpreter loads libraries to %spark and %spark.pyspark but not to  %spark.sql interpreter. So we recommend you to use the first option instead.When your code requires external library, instead of doing download/copy/restart Zeppelin, you can easily do following jobs using %spark.dep interpreter
 .Load libraries recursively from maven repositoryLoad libraries from local filesystemAdd additional maven repositoryAutomatically add libraries to SparkCluster (You can turn off)Dep interpreter leverages Scala environment. So you can write any Scala code here.Note that %spark.dep interpreter should be used before %spark, %spark.pyspark, %spark.sql.Here&amp;#39;s usages.%spark.depz.reset() // clean up previously added artifact and repository// add maven repositoryz.addRepo(&amp;quot;RepoName&amp;quot;).url(&amp;quot;RepoURL&amp;quot;)// add maven snapshot repositoryz.addRepo(&amp;quot;RepoName&amp;quot;).url(&amp;quot;RepoURL&amp;quot;).snapshot()// add credentials for private maven repositoryz.addRepo(&amp;quot;RepoName&amp;quot;).url(&amp;quot;RepoURL&amp;quot;).username(&amp;quot;username&amp;quot;).password(&amp;quot;password&amp;quot;)// add artifact from filesystemz.load(&amp;quot;/path/to.jar&amp;quot;)// add artifact from maven repository, with no dependencyz.load(&amp;quot;g
 roupId:artifactId:version&amp;quot;).excludeAll()// add artifact recursivelyz.load(&amp;quot;groupId:artifactId:version&amp;quot;)// add artifact recursively except comma separated GroupID:ArtifactId listz.load(&amp;quot;groupId:artifactId:version&amp;quot;).exclude(&amp;quot;groupId:artifactId,groupId:artifactId, ...&amp;quot;)// exclude with patternz.load(&amp;quot;groupId:artifactId:version&amp;quot;).exclude(*)z.load(&amp;quot;groupId:artifactId:version&amp;quot;).exclude(&amp;quot;groupId:artifactId:*&amp;quot;)z.load(&amp;quot;groupId:artifactId:version&amp;quot;).exclude(&amp;quot;groupId:*&amp;quot;)// local() skips adding artifact to spark clusters (skipping sc.addJar())z.load(&amp;quot;groupId:artifactId:version&amp;quot;).local()ZeppelinContextZeppelin automatically injects ZeppelinContext as variable z in your Scala/Python environment. ZeppelinContext provides some additional functions and utilities.Object ExchangeZeppelinContext extends map and it&amp;#39;s shared betwe
 en Scala and Python environment.So you can put some objects from Scala and read it from Python, vice versa.  // Put object from scala%sparkval myObject = ...z.put(&amp;quot;objName&amp;quot;, myObject)    # Get object from python%spark.pysparkmyObject = z.get(&amp;quot;objName&amp;quot;)  Form CreationZeppelinContext provides functions for creating forms.In Scala and Python environments, you can create forms programmatically.  %spark/* Create text input form */z.input(&amp;quot;formName&amp;quot;)/* Create text input form with default value */z.input(&amp;quot;formName&amp;quot;, &amp;quot;defaultValue&amp;quot;)/* Create select form */z.select(&amp;quot;formName&amp;quot;, Seq((&amp;quot;option1&amp;quot;, &amp;quot;option1DisplayName&amp;quot;),                         (&amp;quot;option2&amp;quot;, &amp;quot;option2DisplayName&amp;quot;)))/* Create select form with default value*/z.select(&amp;quot;formName&amp;quot;, &amp;quot;option1&amp;quot;, Seq((&amp;quot;option1&amp;quot;, 
 &amp;quot;option1DisplayName&amp;quot;),                                    (&amp;quot;option2&amp;quot;, &amp;quot;option2DisplayName&amp;quot;)))    %spark.pyspark# Create text input formz.input(&amp;quot;formName&amp;quot;)# Create text input form with default valuez.input(&amp;quot;formName&amp;quot;, &amp;quot;defaultValue&amp;quot;)# Create select formz.select(&amp;quot;formName&amp;quot;, [(&amp;quot;option1&amp;quot;, &amp;quot;option1DisplayName&amp;quot;),                      (&amp;quot;option2&amp;quot;, &amp;quot;option2DisplayName&amp;quot;)])# Create select form with default valuez.select(&amp;quot;formName&amp;quot;, [(&amp;quot;option1&amp;quot;, &amp;quot;option1DisplayName&amp;quot;),                      (&amp;quot;option2&amp;quot;, &amp;quot;option2DisplayName&amp;quot;)], &amp;quot;option1&amp;quot;)  In sql environment, you can create form in simple template.%spark.sqlselect * from ${table=defaultTableName} where text like &amp;#39;%${search}%&amp;#39;To lear
 n more about dynamic form, checkout Dynamic Form.Interpreter setting optionYou can choose one of shared, scoped and isolated options wheh you configure Spark interpreter. Spark interpreter creates separated Scala compiler per each notebook but share a single SparkContext in scoped mode (experimental). It creates separated SparkContext per each notebook in isolated mode.Setting up Zeppelin with KerberosLogical setup with Zeppelin, Kerberos Key Distribution Center (KDC), and Spark on YARN:Configuration SetupOn the server that Zeppelin is installed, install Kerberos client modules and configuration, krb5.conf.This is to make the server communicate with KDC.Set SPARK_HOME in [ZEPPELIN_HOME]/conf/zeppelin-env.sh to use spark-submit(Additionally, you might have to set export HADOOP_CONF_DIR=/etc/hadoop/conf)Add the two properties below to Spark configuration ([SPARK_HOME]/conf/spark-defaults.conf):spark.yarn.principalspark.yarn.keytabNOTE: If you do not have permission to access for the a
 bove spark-defaults.conf file, optionally, you can add the above lines to the Spark Interpreter setting through the Interpreter tab in the Zeppelin UI.That&amp;#39;s it. Play with Zeppelin!",
+      "content"  : "&lt;!--Licensed under the Apache License, Version 2.0 (the &quot;License&quot;);you may not use this file except in compliance with the License.You may obtain a copy of the License athttp://www.apache.org/licenses/LICENSE-2.0Unless required by applicable law or agreed to in writing, softwaredistributed under the License is distributed on an &quot;AS IS&quot; BASIS,WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.See the License for the specific language governing permissions andlimitations under the License.--&gt;Spark Interpreter for Apache ZeppelinOverviewApache Spark is a fast and general-purpose cluster computing system.It provides high-level APIs in Java, Scala, Python and R, and an optimized engine that supports general execution graphs.Apache Spark is supported in Zeppelin with Spark interpreter group which consists of below five interpreters.      Name    Class    Description        %spark    SparkInterpreter    Creates a SparkConte
 xt and provides a Scala environment        %spark.pyspark    PySparkInterpreter    Provides a Python environment        %spark.r    SparkRInterpreter    Provides an R environment with SparkR support        %spark.sql    SparkSQLInterpreter    Provides a SQL environment        %spark.dep    DepInterpreter    Dependency loader  ConfigurationThe Spark interpreter can be configured with properties provided by Zeppelin.You can also set other Spark properties which are not listed in the table. For a list of additional properties, refer to Spark Available Properties.      Property    Default    Description        args        Spark commandline args      master    local[*]    Spark master uri.  ex) spark://masterhost:7077      spark.app.name    Zeppelin    The name of spark application.        spark.cores.max        Total number of cores to use.  Empty value uses all available core.        spark.executor.memory     1g    Executor memory per worker instance.  ex) 512m, 32g        zeppelin.dep
 .additionalRemoteRepository    spark-packages,  http://dl.bintray.com/spark-packages/maven,  false;    A list of id,remote-repository-URL,is-snapshot;  for each remote repository.        zeppelin.dep.localrepo    local-repo    Local repository for dependency loader        zeppelin.pyspark.python    python    Python command to run pyspark with        zeppelin.spark.concurrentSQL    false    Execute multiple SQL concurrently if set true.        zeppelin.spark.maxResult    1000    Max number of Spark SQL result to display.        zeppelin.spark.printREPLOutput    true    Print REPL output        zeppelin.spark.useHiveContext    true    Use HiveContext instead of SQLContext if it is true.        zeppelin.spark.importImplicit    true    Import implicits, UDF collection, and sql if set true.  Without any configuration, Spark interpreter works out of box in local mode. But if you want to connect to your Spark cluster, you&amp;#39;ll need to follow below two simple steps.1. Export SPARK_HOM
 EIn conf/zeppelin-env.sh, export SPARK_HOME environment variable with your Spark installation path.For example,export SPARK_HOME=/usr/lib/sparkYou can optionally export HADOOP_CONF_DIR and SPARK_SUBMIT_OPTIONSexport HADOOP_CONF_DIR=/usr/lib/hadoopexport SPARK_SUBMIT_OPTIONS=&amp;quot;--packages com.databricks:spark-csv_2.10:1.2.0&amp;quot;For Windows, ensure you have winutils.exe in %HADOOP_HOME%bin. Please see Problems running Hadoop on Windows for the details.2. Set master in Interpreter menuAfter start Zeppelin, go to Interpreter menu and edit master property in your Spark interpreter setting. The value may vary depending on your Spark cluster deployment type.For example,local[*] in local modespark://master:7077 in standalone clusteryarn-client in Yarn client modemesos://host:5050 in Mesos clusterThat&amp;#39;s it. Zeppelin will work with any version of Spark and any deployment type without rebuilding Zeppelin in this way. For the further information about Spark &amp;amp; Zeppeli
 n version compatibility, please refer to &amp;quot;Available Interpreters&amp;quot; section in Zeppelin download page.Note that without exporting SPARK_HOME, it&amp;#39;s running in local mode with included version of Spark. The included version may vary depending on the build profile.SparkContext, SQLContext, SparkSession, ZeppelinContextSparkContext, SQLContext and ZeppelinContext are automatically created and exposed as variable names sc, sqlContext and z, respectively, in Scala, Python and R environments.Staring from 0.6.1 SparkSession is available as variable spark when you are using Spark 2.x.Note that Scala/Python/R environment shares the same SparkContext, SQLContext and ZeppelinContext instance. Dependency ManagementThere are two ways to load external libraries in Spark interpreter. First is using interpreter setting menu and second is loading Spark properties.1. Setting Dependencies via Interpreter SettingPlease see Dependency Management for the details.2. Loading Spark Pr
 opertiesOnce SPARK_HOME is set in conf/zeppelin-env.sh, Zeppelin uses spark-submit as spark interpreter runner. spark-submit supports two ways to load configurations. The first is command line options such as --master and Zeppelin can pass these options to spark-submit by exporting SPARK_SUBMIT_OPTIONS in conf/zeppelin-env.sh. Second is reading configuration options from SPARK_HOME/conf/spark-defaults.conf. Spark properties that user can set to distribute libraries are:      spark-defaults.conf    SPARK_SUBMIT_OPTIONS    Description        spark.jars    --jars    Comma-separated list of local jars to include on the driver and executor classpaths.        spark.jars.packages    --packages    Comma-separated list of maven coordinates of jars to include on the driver and executor classpaths. Will search the local maven repo, then maven central and any additional remote repositories given by --repositories. The format for the coordinates should be groupId:artifactId:version.        spark
 .files    --files    Comma-separated list of files to be placed in the working directory of each executor.  Here are few examples:SPARK_SUBMIT_OPTIONS in conf/zeppelin-env.shexport SPARK_SUBMIT_OPTIONS=&amp;quot;--packages com.databricks:spark-csv_2.10:1.2.0 --jars /path/mylib1.jar,/path/mylib2.jar --files /path/mylib1.py,/path/mylib2.zip,/path/mylib3.egg&amp;quot;SPARK_HOME/conf/spark-defaults.confspark.jars        /path/mylib1.jar,/path/mylib2.jarspark.jars.packages   com.databricks:spark-csv_2.10:1.2.0spark.files       /path/mylib1.py,/path/mylib2.egg,/path/mylib3.zip3. Dynamic Dependency Loading via %spark.dep interpreterNote: %spark.dep interpreter is deprecated since v0.6.0.%spark.dep interpreter loads libraries to %spark and %spark.pyspark but not to  %spark.sql interpreter. So we recommend you to use the first option instead.When your code requires external library, instead of doing download/copy/restart Zeppelin, you can easily do following jobs using %spark.dep interpreter
 .Load libraries recursively from maven repositoryLoad libraries from local filesystemAdd additional maven repositoryAutomatically add libraries to SparkCluster (You can turn off)Dep interpreter leverages Scala environment. So you can write any Scala code here.Note that %spark.dep interpreter should be used before %spark, %spark.pyspark, %spark.sql.Here&amp;#39;s usages.%spark.depz.reset() // clean up previously added artifact and repository// add maven repositoryz.addRepo(&amp;quot;RepoName&amp;quot;).url(&amp;quot;RepoURL&amp;quot;)// add maven snapshot repositoryz.addRepo(&amp;quot;RepoName&amp;quot;).url(&amp;quot;RepoURL&amp;quot;).snapshot()// add credentials for private maven repositoryz.addRepo(&amp;quot;RepoName&amp;quot;).url(&amp;quot;RepoURL&amp;quot;).username(&amp;quot;username&amp;quot;).password(&amp;quot;password&amp;quot;)// add artifact from filesystemz.load(&amp;quot;/path/to.jar&amp;quot;)// add artifact from maven repository, with no dependencyz.load(&amp;quot;g
 roupId:artifactId:version&amp;quot;).excludeAll()// add artifact recursivelyz.load(&amp;quot;groupId:artifactId:version&amp;quot;)// add artifact recursively except comma separated GroupID:ArtifactId listz.load(&amp;quot;groupId:artifactId:version&amp;quot;).exclude(&amp;quot;groupId:artifactId,groupId:artifactId, ...&amp;quot;)// exclude with patternz.load(&amp;quot;groupId:artifactId:version&amp;quot;).exclude(*)z.load(&amp;quot;groupId:artifactId:version&amp;quot;).exclude(&amp;quot;groupId:artifactId:*&amp;quot;)z.load(&amp;quot;groupId:artifactId:version&amp;quot;).exclude(&amp;quot;groupId:*&amp;quot;)// local() skips adding artifact to spark clusters (skipping sc.addJar())z.load(&amp;quot;groupId:artifactId:version&amp;quot;).local()ZeppelinContextZeppelin automatically injects ZeppelinContext as variable z in your Scala/Python environment. ZeppelinContext provides some additional functions and utilities.Object ExchangeZeppelinContext extends map and it&amp;#39;s shared betwe
 en Scala and Python environment.So you can put some objects from Scala and read it from Python, vice versa.  // Put object from scala%sparkval myObject = ...z.put(&amp;quot;objName&amp;quot;, myObject)    # Get object from python%spark.pysparkmyObject = z.get(&amp;quot;objName&amp;quot;)  Form CreationZeppelinContext provides functions for creating forms.In Scala and Python environments, you can create forms programmatically.  %spark/* Create text input form */z.input(&amp;quot;formName&amp;quot;)/* Create text input form with default value */z.input(&amp;quot;formName&amp;quot;, &amp;quot;defaultValue&amp;quot;)/* Create select form */z.select(&amp;quot;formName&amp;quot;, Seq((&amp;quot;option1&amp;quot;, &amp;quot;option1DisplayName&amp;quot;),                         (&amp;quot;option2&amp;quot;, &amp;quot;option2DisplayName&amp;quot;)))/* Create select form with default value*/z.select(&amp;quot;formName&amp;quot;, &amp;quot;option1&amp;quot;, Seq((&amp;quot;option1&amp;quot;, 
 &amp;quot;option1DisplayName&amp;quot;),                                    (&amp;quot;option2&amp;quot;, &amp;quot;option2DisplayName&amp;quot;)))    %spark.pyspark# Create text input formz.input(&amp;quot;formName&amp;quot;)# Create text input form with default valuez.input(&amp;quot;formName&amp;quot;, &amp;quot;defaultValue&amp;quot;)# Create select formz.select(&amp;quot;formName&amp;quot;, [(&amp;quot;option1&amp;quot;, &amp;quot;option1DisplayName&amp;quot;),                      (&amp;quot;option2&amp;quot;, &amp;quot;option2DisplayName&amp;quot;)])# Create select form with default valuez.select(&amp;quot;formName&amp;quot;, [(&amp;quot;option1&amp;quot;, &amp;quot;option1DisplayName&amp;quot;),                      (&amp;quot;option2&amp;quot;, &amp;quot;option2DisplayName&amp;quot;)], &amp;quot;option1&amp;quot;)  In sql environment, you can create form in simple template.%spark.sqlselect * from ${table=defaultTableName} where text like &amp;#39;%${search}%&amp;#39;To lear
 n more about dynamic form, checkout Dynamic Form.Matplotlib Integration (pyspark)Both the python and pyspark interpreters have built-in support for inline visualization using matplotlib, a popular plotting library for python. More details can be found in the python interpreter documentation, since matplotlib support is identical. More advanced interactive plotting can be done with pyspark through utilizing Zeppelin&amp;#39;s built-in Angular Display System, as shown below:Interpreter setting optionYou can choose one of shared, scoped and isolated options wheh you configure Spark interpreter. Spark interpreter creates separated Scala compiler per each notebook but share a single SparkContext in scoped mode (experimental). It creates separated SparkContext per each notebook in isolated mode.Setting up Zeppelin with KerberosLogical setup with Zeppelin, Kerberos Key Distribution Center (KDC), and Spark on YARN:Configuration SetupOn the server that Zeppelin is installed, install Kerberos
  client modules and configuration, krb5.conf.This is to make the server communicate with KDC.Set SPARK_HOME in [ZEPPELIN_HOME]/conf/zeppelin-env.sh to use spark-submit(Additionally, you might have to set export HADOOP_CONF_DIR=/etc/hadoop/conf)Add the two properties below to Spark configuration ([SPARK_HOME]/conf/spark-defaults.conf):spark.yarn.principalspark.yarn.keytabNOTE: If you do not have permission to access for the above spark-defaults.conf file, optionally, you can add the above lines to the Spark Interpreter setting through the Interpreter tab in the Zeppelin UI.That&amp;#39;s it. Play with Zeppelin!",
       "url": " /interpreter/spark.html",
       "group": "interpreter",
       "excerpt": "Apache Spark is a fast and general-purpose cluster computing system. It provides high-level APIs in Java, Scala, Python and R, and an optimized engine that supports general execution engine."
@@ -583,7 +583,7 @@
 
     "/security/shiroauthentication.html": {
       "title": "Apache Shiro Authentication for Apache Zeppelin",
-      "content"  : "&lt;!--Licensed under the Apache License, Version 2.0 (the &quot;License&quot;);you may not use this file except in compliance with the License.You may obtain a copy of the License athttp://www.apache.org/licenses/LICENSE-2.0Unless required by applicable law or agreed to in writing, softwaredistributed under the License is distributed on an &quot;AS IS&quot; BASIS,WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.See the License for the specific language governing permissions andlimitations under the License.--&gt;{% include JB/setup %}# Apache Shiro authentication for Apache Zeppelin## Overview[Apache Shiro](http://shiro.apache.org/) is a powerful and easy-to-use Java security framework that performs authentication, authorization, cryptography, and session management. In this documentation, we will explain step by step how Shiro works for Zeppelin notebook authentication.When you connect to Apache Zeppelin, you will be asked to enter your c
 redentials. Once you logged in, then you have access to all notes including other user&#39;s notes.## Security SetupYou can setup **Zeppelin notebook authentication** in some simple steps.### 1. Enable ShiroBy default in `conf`, you will find `shiro.ini.template`, this file is used as an example and it is strongly recommendedto create a `shiro.ini` file by doing the following command line `cp conf/shiro.ini.template conf/shiro.ini`.### 2. Secure the HTTP channelTo secure the HTTP channel, you have to change both **anon** and **authc** settings in `conf/shiro.ini`. In here, **anon** means &quot;the access is anonymous&quot; and **authc** means &quot;formed auth security&quot;.The default status of them is```/** = anon#/** = authc```Deactivate the line &quot;/** = anon&quot; and activate the line &quot;/** = authc&quot; in `conf/shiro.ini` file.```#/** = anon/** = authc```For the further information about  `shiro.ini` file format, please refer to [Shiro Configuration](http://shiro.apa
 che.org/configuration.html#Configuration-INISections).### 3. Secure the Websocket channelSet to property **zeppelin.anonymous.allowed** to **false** in `conf/zeppelin-site.xml`. If you don&#39;t have this file yet, just copy `conf/zeppelin-site.xml.template` to `conf/zeppelin-site.xml`.### 4. Start Zeppelin```bin/zeppelin-daemon.sh start (or restart)```Then you can browse Zeppelin at [http://localhost:8080](http://localhost:8080).### 5. LoginFinally, you can login using one of the below **username/password** combinations.```[users]admin = password1, adminuser1 = password2, role1, role2user2 = password3, role3user3 = password4, role2```You can set the roles for each users next to the password.## Groups and permissions (optional)In case you want to leverage user groups and permissions, use one of the following configuration for LDAP or AD under `[main]` segment in `shiro.ini`.```activeDirectoryRealm = org.apache.zeppelin.server.ActiveDirectoryGroupRealmactiveDirectoryRealm.systemUsern
 ame = userNameAactiveDirectoryRealm.systemPassword = passwordAactiveDirectoryRealm.searchBase = CN=Users,DC=SOME_GROUP,DC=COMPANY,DC=COMactiveDirectoryRealm.url = ldap://ldap.test.com:389activeDirectoryRealm.groupRolesMap = &quot;CN=aGroupName,OU=groups,DC=SOME_GROUP,DC=COMPANY,DC=COM&quot;:&quot;group1&quot;activeDirectoryRealm.authorizationCachingEnabled = falseldapRealm = org.apache.zeppelin.server.LdapGroupRealm# search base for ldap groups (only relevant for LdapGroupRealm):ldapRealm.contextFactory.environment[ldap.searchBase] = dc=COMPANY,dc=COMldapRealm.contextFactory.url = ldap://ldap.test.com:389ldapRealm.userDnTemplate = uid={0},ou=Users,dc=COMPANY,dc=COMldapRealm.contextFactory.authenticationMechanism = SIMPLE```also define roles/groups that you want to have in system, like below;```[roles]admin = *hr = *finance = *group1 = *```## Configure Realm (optional)Realms are responsible for authentication and authorization in Apache Zeppelin. By default, Apache Zeppelin uses [Ini
 Realm](https://shiro.apache.org/static/latest/apidocs/org/apache/shiro/realm/text/IniRealm.html) (users and groups are configurable in `conf/shiro.ini` file under `[user]` and `[group]` section). You can also leverage Shiro Realms like [JndiLdapRealm](https://shiro.apache.org/static/latest/apidocs/org/apache/shiro/realm/ldap/JndiLdapRealm.html), [JdbcRealm](https://shiro.apache.org/static/latest/apidocs/org/apache/shiro/realm/jdbc/JdbcRealm.html) or create [our own](https://shiro.apache.org/static/latest/apidocs/org/apache/shiro/realm/AuthorizingRealm.html).To learn more about Apache Shiro Realm, please check [this documentation](http://shiro.apache.org/realm.html).We also provide community custom Realms.### Active Directory```activeDirectoryRealm = org.apache.zeppelin.server.ActiveDirectoryGroupRealmactiveDirectoryRealm.systemUsername = userNameAactiveDirectoryRealm.systemPassword = passwordAactiveDirectoryRealm.hadoopSecurityCredentialPath = jceks://file/user/zeppelin/conf/zeppeli
 n.jceksactiveDirectoryRealm.searchBase = CN=Users,DC=SOME_GROUP,DC=COMPANY,DC=COMactiveDirectoryRealm.url = ldap://ldap.test.com:389activeDirectoryRealm.groupRolesMap = &quot;CN=aGroupName,OU=groups,DC=SOME_GROUP,DC=COMPANY,DC=COM&quot;:&quot;group1&quot;activeDirectoryRealm.authorizationCachingEnabled = false```Also instead of specifying systemPassword in clear text in shiro.ini administrator can choose to specify the same in &quot;hadoop credential&quot;.Create a keystore file using the hadoop credential commandline, for this the hadoop commons should be in the classpath`hadoop credential create activeDirectoryRealm.systempassword -provider jceks://file/user/zeppelin/conf/zeppelin.jceks`Change the following values in the Shiro.ini file, and uncomment the line:`activeDirectoryRealm.hadoopSecurityCredentialPath = jceks://file/user/zeppelin/conf/zeppelin.jceks`### LDAP```ldapRealm = org.apache.zeppelin.server.LdapGroupRealm# search base for ldap groups (only relevant for LdapGroupRea
 lm):ldapRealm.contextFactory.environment[ldap.searchBase] = dc=COMPANY,dc=COMldapRealm.contextFactory.url = ldap://ldap.test.com:389ldapRealm.userDnTemplate = uid={0},ou=Users,dc=COMPANY,dc=COMldapRealm.contextFactory.authenticationMechanism = SIMPLE```### ZeppelinHub[ZeppelinHub](https://www.zeppelinhub.com) is a service that synchronize your Apache Zeppelin notebooks and enables you to collaborate easily.To enable login with your ZeppelinHub credential, apply the following change in `conf/shiro.ini` under `[main]` section.```### A sample for configuring ZeppelinHub RealmzeppelinHubRealm = org.apache.zeppelin.realm.ZeppelinHubRealm## Url of ZeppelinHubzeppelinHubRealm.zeppelinhubUrl = https://www.zeppelinhub.comsecurityManager.realms = $zeppelinHubRealm```&gt; Note: ZeppelinHub is not releated to apache Zeppelin project.## Secure your Zeppelin information (optional)By default, anyone who defined in `[users]` can share **Interpreter Setting**, **Credential** and **Configuration** in
 formation in Apache Zeppelin.Sometimes you might want to hide these information for your use case.Since Shiro provides **url-based security**, you can hide the information by commenting or uncommenting these below lines in `conf/shiro.ini`.```[urls]/api/interpreter/** = authc, roles[admin]/api/configurations/** = authc, roles[admin]/api/credential/** = authc, roles[admin]```In this case, only who have `admin` role can see **Interpreter Setting**, **Credential** and **Configuration** information.If you want to grant this permission to other users, you can change **roles[ ]** as you defined at `[users]` section.&gt; **NOTE :** All of the above configurations are defined in the `conf/shiro.ini` file. This documentation is originally from [SECURITY-README.md](https://github.com/apache/zeppelin/blob/master/SECURITY-README.md).",
+      "content"  : "&lt;!--Licensed under the Apache License, Version 2.0 (the &quot;License&quot;);you may not use this file except in compliance with the License.You may obtain a copy of the License athttp://www.apache.org/licenses/LICENSE-2.0Unless required by applicable law or agreed to in writing, softwaredistributed under the License is distributed on an &quot;AS IS&quot; BASIS,WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.See the License for the specific language governing permissions andlimitations under the License.--&gt;{% include JB/setup %}# Apache Shiro authentication for Apache Zeppelin## Overview[Apache Shiro](http://shiro.apache.org/) is a powerful and easy-to-use Java security framework that performs authentication, authorization, cryptography, and session management. In this documentation, we will explain step by step how Shiro works for Zeppelin notebook authentication.When you connect to Apache Zeppelin, you will be asked to enter your c
 redentials. Once you logged in, then you have access to all notes including other user&#39;s notes.## Security SetupYou can setup **Zeppelin notebook authentication** in some simple steps.### 1. Enable ShiroBy default in `conf`, you will find `shiro.ini.template`, this file is used as an example and it is strongly recommendedto create a `shiro.ini` file by doing the following command line `cp conf/shiro.ini.template conf/shiro.ini`.### 2. Secure the HTTP channelTo secure the HTTP channel, you have to change both **anon** and **authc** settings in `conf/shiro.ini`. In here, **anon** means &quot;the access is anonymous&quot; and **authc** means &quot;formed auth security&quot;.The default status of them is```/** = anon#/** = authc```Deactivate the line &quot;/** = anon&quot; and activate the line &quot;/** = authc&quot; in `conf/shiro.ini` file.```#/** = anon/** = authc```For the further information about  `shiro.ini` file format, please refer to [Shiro Configuration](http://shiro.apa
 che.org/configuration.html#Configuration-INISections).### 3. Secure the Websocket channelSet to property **zeppelin.anonymous.allowed** to **false** in `conf/zeppelin-site.xml`. If you don&#39;t have this file yet, just copy `conf/zeppelin-site.xml.template` to `conf/zeppelin-site.xml`.### 4. Start Zeppelin```bin/zeppelin-daemon.sh start (or restart)```Then you can browse Zeppelin at [http://localhost:8080](http://localhost:8080).### 5. LoginFinally, you can login using one of the below **username/password** combinations.```[users]admin = password1, adminuser1 = password2, role1, role2user2 = password3, role3user3 = password4, role2```You can set the roles for each users next to the password.## Groups and permissions (optional)In case you want to leverage user groups and permissions, use one of the following configuration for LDAP or AD under `[main]` segment in `shiro.ini`.```activeDirectoryRealm = org.apache.zeppelin.server.ActiveDirectoryGroupRealmactiveDirectoryRealm.systemUsern
 ame = userNameAactiveDirectoryRealm.systemPassword = passwordAactiveDirectoryRealm.searchBase = CN=Users,DC=SOME_GROUP,DC=COMPANY,DC=COMactiveDirectoryRealm.url = ldap://ldap.test.com:389activeDirectoryRealm.groupRolesMap = &quot;CN=aGroupName,OU=groups,DC=SOME_GROUP,DC=COMPANY,DC=COM&quot;:&quot;group1&quot;activeDirectoryRealm.authorizationCachingEnabled = falseldapRealm = org.apache.zeppelin.server.LdapGroupRealm# search base for ldap groups (only relevant for LdapGroupRealm):ldapRealm.contextFactory.environment[ldap.searchBase] = dc=COMPANY,dc=COMldapRealm.contextFactory.url = ldap://ldap.test.com:389ldapRealm.userDnTemplate = uid={0},ou=Users,dc=COMPANY,dc=COMldapRealm.contextFactory.authenticationMechanism = SIMPLE```also define roles/groups that you want to have in system, like below;```[roles]admin = *hr = *finance = *group1 = *```## Configure Realm (optional)Realms are responsible for authentication and authorization in Apache Zeppelin. By default, Apache Zeppelin uses [Ini
 Realm](https://shiro.apache.org/static/latest/apidocs/org/apache/shiro/realm/text/IniRealm.html) (users and groups are configurable in `conf/shiro.ini` file under `[user]` and `[group]` section). You can also leverage Shiro Realms like [JndiLdapRealm](https://shiro.apache.org/static/latest/apidocs/org/apache/shiro/realm/ldap/JndiLdapRealm.html), [JdbcRealm](https://shiro.apache.org/static/latest/apidocs/org/apache/shiro/realm/jdbc/JdbcRealm.html) or create [our own](https://shiro.apache.org/static/latest/apidocs/org/apache/shiro/realm/AuthorizingRealm.html).To learn more about Apache Shiro Realm, please check [this documentation](http://shiro.apache.org/realm.html).We also provide community custom Realms.### Active Directory```activeDirectoryRealm = org.apache.zeppelin.server.ActiveDirectoryGroupRealmactiveDirectoryRealm.systemUsername = userNameAactiveDirectoryRealm.systemPassword = passwordAactiveDirectoryRealm.hadoopSecurityCredentialPath = jceks://file/user/zeppelin/conf/zeppeli
 n.jceksactiveDirectoryRealm.searchBase = CN=Users,DC=SOME_GROUP,DC=COMPANY,DC=COMactiveDirectoryRealm.url = ldap://ldap.test.com:389activeDirectoryRealm.groupRolesMap = &quot;CN=aGroupName,OU=groups,DC=SOME_GROUP,DC=COMPANY,DC=COM&quot;:&quot;group1&quot;activeDirectoryRealm.authorizationCachingEnabled = false```Also instead of specifying systemPassword in clear text in shiro.ini administrator can choose to specify the same in &quot;hadoop credential&quot;.Create a keystore file using the hadoop credential commandline, for this the hadoop commons should be in the classpath`hadoop credential create activeDirectoryRealm.systempassword -provider jceks://file/user/zeppelin/conf/zeppelin.jceks`Change the following values in the Shiro.ini file, and uncomment the line:`activeDirectoryRealm.hadoopSecurityCredentialPath = jceks://file/user/zeppelin/conf/zeppelin.jceks`### LDAP```ldapRealm = org.apache.zeppelin.server.LdapGroupRealm# search base for ldap groups (only relevant for LdapGroupRea
 lm):ldapRealm.contextFactory.environment[ldap.searchBase] = dc=COMPANY,dc=COMldapRealm.contextFactory.url = ldap://ldap.test.com:389ldapRealm.userDnTemplate = uid={0},ou=Users,dc=COMPANY,dc=COMldapRealm.contextFactory.authenticationMechanism = SIMPLE```### PAM[PAM](https://en.wikipedia.org/wiki/Pluggable_authentication_module) authentication support allows the reuse of existing authentication moduls on the host where Zeppelin is running. On a typical system modules are configured per service for example sshd, passwd, etc. under `/etc/pam.d/`. You caneither reuse one of these services or create your own for Zeppelin. Activiting PAM authentication requires two parameters: 1. realm: The Shiro realm being used 2. service: The service configured under `/etc/pam.d/` to be used. The name here needs to be the same as the file name under `/etc/pam.d/` ```[main] pamRealm=org.apache.zeppelin.realm.PamRealm pamRealm.service=sshd```### ZeppelinHub[ZeppelinHub](https://www.zeppelinhub.com) is a s
 ervice that synchronize your Apache Zeppelin notebooks and enables you to collaborate easily.To enable login with your ZeppelinHub credential, apply the following change in `conf/shiro.ini` under `[main]` section.```### A sample for configuring ZeppelinHub RealmzeppelinHubRealm = org.apache.zeppelin.realm.ZeppelinHubRealm## Url of ZeppelinHubzeppelinHubRealm.zeppelinhubUrl = https://www.zeppelinhub.comsecurityManager.realms = $zeppelinHubRealm```&gt; Note: ZeppelinHub is not releated to apache Zeppelin project.## Secure your Zeppelin information (optional)By default, anyone who defined in `[users]` can share **Interpreter Setting**, **Credential** and **Configuration** information in Apache Zeppelin.Sometimes you might want to hide these information for your use case.Since Shiro provides **url-based security**, you can hide the information by commenting or uncommenting these below lines in `conf/shiro.ini`.```[urls]/api/interpreter/** = authc, roles[admin]/api/configurations/** = au
 thc, roles[admin]/api/credential/** = authc, roles[admin]```In this case, only who have `admin` role can see **Interpreter Setting**, **Credential** and **Configuration** information.If you want to grant this permission to other users, you can change **roles[ ]** as you defined at `[users]` section.&gt; **NOTE :** All of the above configurations are defined in the `conf/shiro.ini` file. This documentation is originally from [SECURITY-README.md](https://github.com/apache/zeppelin/blob/master/SECURITY-README.md).",
       "url": " /security/shiroauthentication.html",
       "group": "security",
       "excerpt": "Apache Shiro is a powerful and easy-to-use Java security framework that performs authentication, authorization, cryptography, and session management. This document explains step by step how Shiro can be used for Zeppelin notebook authentication."

Modified: zeppelin/site/docs/0.7.0-SNAPSHOT/security/shiroauthentication.html
URL: http://svn.apache.org/viewvc/zeppelin/site/docs/0.7.0-SNAPSHOT/security/shiroauthentication.html?rev=1768714&r1=1768713&r2=1768714&view=diff
==============================================================================
--- zeppelin/site/docs/0.7.0-SNAPSHOT/security/shiroauthentication.html (original)
+++ zeppelin/site/docs/0.7.0-SNAPSHOT/security/shiroauthentication.html Tue Nov  8 15:33:30 2016
@@ -314,6 +314,17 @@ ldapRealm.contextFactory.url = ldap://ld
 ldapRealm.userDnTemplate = uid={0},ou=Users,dc=COMPANY,dc=COM
 ldapRealm.contextFactory.authenticationMechanism = SIMPLE
 </code></pre></div>
+<h3>PAM</h3>
+
+<p><a href="https://en.wikipedia.org/wiki/Pluggable_authentication_module">PAM</a> authentication support allows the reuse of existing authentication 
+moduls on the host where Zeppelin is running. On a typical system modules are configured per service for example sshd, passwd, etc. under <code>/etc/pam.d/</code>. You can
+either reuse one of these services or create your own for Zeppelin. Activiting PAM authentication requires two parameters:
+ 1. realm: The Shiro realm being used
+ 2. service: The service configured under <code>/etc/pam.d/</code> to be used. The name here needs to be the same as the file name under <code>/etc/pam.d/</code></p>
+<div class="highlight"><pre><code class="text language-text" data-lang="text">[main]
+ pamRealm=org.apache.zeppelin.realm.PamRealm
+ pamRealm.service=sshd
+</code></pre></div>
 <h3>ZeppelinHub</h3>
 
 <p><a href="https://www.zeppelinhub.com">ZeppelinHub</a> is a service that synchronize your Apache Zeppelin notebooks and enables you to collaborate easily.</p>



Mime
View raw message