Return-Path: X-Original-To: archive-asf-public-internal@cust-asf2.ponee.io Delivered-To: archive-asf-public-internal@cust-asf2.ponee.io Received: from cust-asf.ponee.io (cust-asf.ponee.io [163.172.22.183]) by cust-asf2.ponee.io (Postfix) with ESMTP id 26E82200B25 for ; Wed, 8 Jun 2016 21:02:47 +0200 (CEST) Received: by cust-asf.ponee.io (Postfix) id 25CEA160A2E; Wed, 8 Jun 2016 19:02:47 +0000 (UTC) Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by cust-asf.ponee.io (Postfix) with SMTP id F24A5160A0E for ; Wed, 8 Jun 2016 21:02:44 +0200 (CEST) Received: (qmail 70018 invoked by uid 500); 8 Jun 2016 19:02:44 -0000 Mailing-List: contact commits-help@zeppelin.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@zeppelin.apache.org Delivered-To: mailing list commits@zeppelin.apache.org Received: (qmail 70009 invoked by uid 99); 8 Jun 2016 19:02:44 -0000 Received: from pnap-us-west-generic-nat.apache.org (HELO spamd1-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 08 Jun 2016 19:02:44 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd1-us-west.apache.org (ASF Mail Server at spamd1-us-west.apache.org) with ESMTP id 8D3C7C0651 for ; Wed, 8 Jun 2016 19:02:43 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd1-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: 0.374 X-Spam-Level: X-Spam-Status: No, score=0.374 tagged_above=-999 required=6.31 tests=[KAM_ASCII_DIVIDERS=0.8, KAM_LAZY_DOMAIN_SECURITY=1, RP_MATCHES_RCVD=-1.426] autolearn=disabled Received: from mx1-lw-eu.apache.org ([10.40.0.8]) by localhost (spamd1-us-west.apache.org [10.40.0.7]) (amavisd-new, port 10024) with ESMTP id vDkIZp7xJgut for ; Wed, 8 Jun 2016 19:02:38 +0000 (UTC) Received: from mailrelay1-us-west.apache.org (mailrelay1-us-west.apache.org [209.188.14.139]) by mx1-lw-eu.apache.org (ASF Mail Server at mx1-lw-eu.apache.org) with ESMTP id E979B5FB3F for ; Wed, 8 Jun 2016 19:02:35 +0000 (UTC) Received: from svn01-us-west.apache.org (svn.apache.org [10.41.0.6]) by mailrelay1-us-west.apache.org (ASF Mail Server at mailrelay1-us-west.apache.org) with ESMTP id 1C4D8E0C18 for ; Wed, 8 Jun 2016 19:02:34 +0000 (UTC) Received: from svn01-us-west.apache.org (localhost [127.0.0.1]) by svn01-us-west.apache.org (ASF Mail Server at svn01-us-west.apache.org) with ESMTP id 86DC13A0E70 for ; Wed, 8 Jun 2016 19:02:33 +0000 (UTC) Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Subject: svn commit: r1747434 [3/4] - in /incubator/zeppelin/site/docs/0.6.0-SNAPSHOT: ./ assets/themes/zeppelin/css/ development/ displaysystem/ install/ interpreter/ manual/ quickstart/ rest-api/ security/ storage/ tutorial/ ui_layout/ Date: Wed, 08 Jun 2016 19:02:32 -0000 To: commits@zeppelin.incubator.apache.org From: minalee@apache.org X-Mailer: svnmailer-1.0.9 Message-Id: <20160608190233.86DC13A0E70@svn01-us-west.apache.org> archived-at: Wed, 08 Jun 2016 19:02:47 -0000 Modified: incubator/zeppelin/site/docs/0.6.0-SNAPSHOT/manual/notebookashomepage.html URL: http://svn.apache.org/viewvc/incubator/zeppelin/site/docs/0.6.0-SNAPSHOT/manual/notebookashomepage.html?rev=1747434&r1=1747433&r2=1747434&view=diff ============================================================================== --- incubator/zeppelin/site/docs/0.6.0-SNAPSHOT/manual/notebookashomepage.html (original) +++ incubator/zeppelin/site/docs/0.6.0-SNAPSHOT/manual/notebookashomepage.html Wed Jun 8 19:02:31 2016 @@ -63,6 +63,8 @@
  • Overview
    • +
  • Explore Zeppelin UI
    • +
  • Tutorial
  • Install
    • @@ -71,11 +73,6 @@
  • Upgrade Version
    • - -
  • Tutorial
    • - -
  • UI Layout
    • -
  • Dynamic Form
  • Publish your Paragraph
    • @@ -100,7 +97,7 @@
  • Livy
  • Markdown
  • Python
    • -
  • Postgresql, hawq
    • +
  • Postgresql, HAWQ
  • R
  • Scalding
  • Shell
    • @@ -114,8 +111,8 @@
  • Display System
      -
    • Text
      • -
    • Html
      • +
    • Text
      • +
    • Html
    • Table
    • Angular (backend API)
    • Angular (frontend API)
      • @@ -157,7 +154,7 @@ -
      +
    • Overview
      • +
    • Explore Zeppelin UI
      • +
    • Tutorial
    • Install
      • @@ -71,11 +73,6 @@
    • Upgrade Version
      • - -
    • Tutorial
      • - -
    • UI Layout
      • -
    • Dynamic Form
    • Publish your Paragraph
      • @@ -100,7 +97,7 @@
    • Livy
    • Markdown
    • Python
      • -
    • Postgresql, hawq
      • +
    • Postgresql, HAWQ
    • R
    • Scalding
    • Shell
      • @@ -114,8 +111,8 @@
    • Display System
      • -
      +
    • Overview
      • +
    • Explore Zeppelin UI
      • +
    • Tutorial
    • Install
      • @@ -71,11 +73,6 @@
    • Upgrade Version
      • - -
    • Tutorial
      • - -
    • UI Layout
      • -
    • Dynamic Form
    • Publish your Paragraph
      • @@ -100,7 +97,7 @@
    • Livy
    • Markdown
    • Python
      • -
    • Postgresql, hawq
      • +
    • Postgresql, HAWQ
    • R
    • Scalding
    • Shell
      • @@ -114,8 +111,8 @@
    • Display System
      • -
      +
      + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
      + + + +
      +
      + + +

      Home Page

      + +

      The first time you connect to Zeppelin, you'll land at the main page similar to the below screen capture

      + +

      + +

      On the left of the page are listed all existing notes. Those notes are stored by default in the $ZEPPELIN_HOME/notebook folder.

      + +

      You can filter them by name using the input text form. You can also create an new note, refresh the list of existing notes +(in case you manually copy them into the $ZEPPELIN_HOME/notebook folder) and import a note.

      + +

      + +

      When clicking on Import Note link, a new dialog open. From there you can import your note from local disk or from a remote location +if you provide the URL.

      + +

      + +

      By default, the name of the imported note is the same as the original note but you can override it by providing a new name.

      + +


      + +

      Menus

      + +

      1. Notebook

      + +

      The Notebook menu proposes almost the same features as the note management section in the home page. From the drop-down menu you can:

      + +
        +
      1. Open a selected note
        2. +
      2. Filter node by name
        3. +
      3. Create a new note
        4. +
      + +

      + +

      2. Interpreter

      + +

      In this menu you can:

      + +
        +
      1. Configure existing interpreter instance
        2. +
      2. Add/remove interpreter instances
        3. +
      + +

      + +

      3. Configuration

      + +

      This menu displays all the Zeppelin configuration that are set in the config file $ZEPPELIN_HOME/conf/zeppelin-site.xml

      + +

      + +


      + +

      Note Layout

      + +

      Each Zeppelin note is composed of 1 .. N paragraphs. The note can be viewed as a paragraph container.

      + +

      + +

      Paragraph

      + +

      Each paragraph consists of 2 sections: code section where you put your source code and result section where you can see the result of the code execution.

      + +

      + +

      On the top-right corner of each paragraph there are some commands to:

      + +
        +
      • execute the paragraph code
        • +
      • hide/show code section
        • +
      • hide/show result section
        • +
      • configure the paragraph
        • +
      + +

      To configure the paragraph, just click on the gear icon:

      + +

      + +

      From this dialog, you can (in descending order):

      + +
        +
      • find the paragraph id ( 20150924-163507_134879501 )
        • +
      • control paragraph width. Since Zeppelin is using the grid system of Twitter Bootstrap, each paragraph width can be changed from 1 to 12
        • +
      • move the paragraph 1 level up
        • +
      • move the paragraph 1 level down
        • +
      • create a new paragraph
        • +
      • change paragraph title
        • +
      • show/hide line number in the code section
        • +
      • disable the run button for this paragraph
        • +
      • export the current paragraph as an iframe and open the iframe in a new window
        • +
      • clear the result section
        • +
      • delete the current paragraph
        • +
      + +

      Note toolbar

      + +

      At the top of the note, you can find a toolbar which exposes command buttons as well as configuration, security and display options.

      + +

      + +

      On the far right is displayed the note name, just click on it to reveal the input form and update it.

      + +

      In the middle of the toolbar you can find the command buttons:

      + +
        +
      • execute all the paragraphs sequentially, in their display order
        • +
      • hide/show code section of all paragraphs
        • +
      • hide/show result section of all paragraphs
        • +
      • clear the result section of all paragraphs
        • +
      • clone the current note
        • +
      • export the current note to a JSON file. _Please note that the code section and result section of all paragraphs will be exported. If you have heavy data in the result section of some paragraphs, it is recommended to clean them before exporting
        • +
      • commit the current node content
        • +
      • delete the note
        • +
      • schedule the execution of all paragraph using a CRON syntax
        • +
      + +

      + +

      On the right of the note tool bar you can find configuration icons:

      + +
        +
      • display all the keyboard shorcuts
        • +
      • configure the interpreters binding to the current note
        • +
      • configure the note permissions
        • +
      • switch the node display mode between default, simple and report
        • +
      + +

      + +
      +
      + + +
      +
      + +
      +
      + + + + + + + + + + + Propchange: incubator/zeppelin/site/docs/0.6.0-SNAPSHOT/quickstart/explorezeppelinui.html ------------------------------------------------------------------------------ svn:eol-style = native Added: incubator/zeppelin/site/docs/0.6.0-SNAPSHOT/quickstart/tutorial.html URL: http://svn.apache.org/viewvc/incubator/zeppelin/site/docs/0.6.0-SNAPSHOT/quickstart/tutorial.html?rev=1747434&view=auto ============================================================================== --- incubator/zeppelin/site/docs/0.6.0-SNAPSHOT/quickstart/tutorial.html (added) +++ incubator/zeppelin/site/docs/0.6.0-SNAPSHOT/quickstart/tutorial.html Wed Jun 8 19:02:31 2016 @@ -0,0 +1,358 @@ + + + + + + Tutorial + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
      + + + +
      +
      + + +

      Zeppelin Tutorial

      + +

      This tutorial walks you through some of the fundamental Zeppelin concepts. We will assume you have already installed Zeppelin. If not, please see here first.

      + +

      Current main backend processing engine of Zeppelin is Apache Spark. If you're new to this system, you might want to start by getting an idea of how it processes data to get the most out of Zeppelin.

      + +


      + +

      Tutorial with Local File

      + +

      1. Data Refine

      + +

      Before you start Zeppelin tutorial, you will need to download bank.zip.

      + +

      First, to transform csv format data into RDD of Bank objects, run following script. This will also remove header using filter function.

      +
      val bankText = sc.textFile("yourPath/bank/bank-full.csv")
+
+case class Bank(age:Integer, job:String, marital : String, education : String, balance : Integer)
+
+// split each line, filter out header (starts with "age"), and map it into Bank case class
+val bank = bankText.map(s=>s.split(";")).filter(s=>s(0)!="\"age\"").map(
+    s=>Bank(s(0).toInt, 
+            s(1).replaceAll("\"", ""),
+            s(2).replaceAll("\"", ""),
+            s(3).replaceAll("\"", ""),
+            s(5).replaceAll("\"", "").toInt
+        )
+)
+
+// convert to DataFrame and create temporal table
+bank.toDF().registerTempTable("bank")
+
      +

      2. Data Retrieval

      + +

      Suppose we want to see age distribution from bank. To do this, run:

      +
      %sql select age, count(1) from bank where age < 30 group by age order by age
+
      +

      You can make input box for setting age condition by replacing 30 with ${maxAge=30}.

      +
      %sql select age, count(1) from bank where age < ${maxAge=30} group by age order by age
+
      +

      Now we want to see age distribution with certain marital status and add combo box to select marital status. Run:

      +
      %sql select age, count(1) from bank where marital="${marital=single,single|divorced|married}" group by age order by age
+
      +


      + +

      Tutorial with Streaming Data

      + +

      1. Data Refine

      + +

      Since this tutorial is based on Twitter's sample tweet stream, you must configure authentication with a Twitter account. To do this, take a look at Twitter Credential Setup. After you get API keys, you should fill out credential related values(apiKey, apiSecret, accessToken, accessTokenSecret) with your API keys on following script.

      + +

      This will create a RDD of Tweet objects and register these stream data as a table:

      +
      import org.apache.spark.streaming._
+import org.apache.spark.streaming.twitter._
+import org.apache.spark.storage.StorageLevel
+import scala.io.Source
+import scala.collection.mutable.HashMap
+import java.io.File
+import org.apache.log4j.Logger
+import org.apache.log4j.Level
+import sys.process.stringSeqToProcess
+
+/** Configures the Oauth Credentials for accessing Twitter */
+def configureTwitterCredentials(apiKey: String, apiSecret: String, accessToken: String, accessTokenSecret: String) {
+  val configs = new HashMap[String, String] ++= Seq(
+    "apiKey" -> apiKey, "apiSecret" -> apiSecret, "accessToken" -> accessToken, "accessTokenSecret" -> accessTokenSecret)
+  println("Configuring Twitter OAuth")
+  configs.foreach{ case(key, value) =>
+    if (value.trim.isEmpty) {
+      throw new Exception("Error setting authentication - value for " + key + " not set")
+    }
+    val fullKey = "twitter4j.oauth." + key.replace("api", "consumer")
+    System.setProperty(fullKey, value.trim)
+    println("\tProperty " + fullKey + " set as [" + value.trim + "]")
+  }
+  println()
+}
+
+// Configure Twitter credentials
+val apiKey = "xxxxxxxxxxxxxxxxxxxxxxxxx"
+val apiSecret = "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx"
+val accessToken = "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx"
+val accessTokenSecret = "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx"
+configureTwitterCredentials(apiKey, apiSecret, accessToken, accessTokenSecret)
+
+import org.apache.spark.streaming.twitter._
+val ssc = new StreamingContext(sc, Seconds(2))
+val tweets = TwitterUtils.createStream(ssc, None)
+val twt = tweets.window(Seconds(60))
+
+case class Tweet(createdAt:Long, text:String)
+twt.map(status=>
+  Tweet(status.getCreatedAt().getTime()/1000, status.getText())
+).foreachRDD(rdd=>
+  // Below line works only in spark 1.3.0.
+  // For spark 1.1.x and spark 1.2.x,
+  // use rdd.registerTempTable("tweets") instead.
+  rdd.toDF().registerAsTable("tweets")
+)
+
+twt.print
+
+ssc.start()
+
      +

      2. Data Retrieval

      + +

      For each following script, every time you click run button you will see different result since it is based on real-time data.

      + +

      Let's begin by extracting maximum 10 tweets which contain the word girl.

      +
      %sql select * from tweets where text like '%girl%' limit 10
+
      +

      This time suppose we want to see how many tweets have been created per sec during last 60 sec. To do this, run:

      +
      %sql select createdAt, count(1) from tweets group by createdAt order by createdAt
+
      +

      You can make user-defined function and use it in Spark SQL. Let's try it by making function named sentiment. This function will return one of the three attitudes( positive, negative, neutral ) towards the parameter.

      +
      def sentiment(s:String) : String = {
+    val positive = Array("like", "love", "good", "great", "happy", "cool", "the", "one", "that")
+    val negative = Array("hate", "bad", "stupid", "is")
+
+    var st = 0;
+
+    val words = s.split(" ")    
+    positive.foreach(p =>
+        words.foreach(w =>
+            if(p==w) st = st+1
+        )
+    )
+
+    negative.foreach(p=>
+        words.foreach(w=>
+            if(p==w) st = st-1
+        )
+    )
+    if(st>0)
+        "positivie"
+    else if(st<0)
+        "negative"
+    else
+        "neutral"
+}
+
+// Below line works only in spark 1.3.0.
+// For spark 1.1.x and spark 1.2.x,
+// use sqlc.registerFunction("sentiment", sentiment _) instead.
+sqlc.udf.register("sentiment", sentiment _)
+
      +

      To check how people think about girls using sentiment function we've made above, run this:

      +
      %sql select sentiment(text), count(1) from tweets where text like '%girl%' group by sentiment(text)
+
      +
      +
      + + +
      +
      + +
      +
      + + + + + + + + + + + Propchange: incubator/zeppelin/site/docs/0.6.0-SNAPSHOT/quickstart/tutorial.html ------------------------------------------------------------------------------ svn:eol-style = native Modified: incubator/zeppelin/site/docs/0.6.0-SNAPSHOT/rest-api/rest-configuration.html URL: http://svn.apache.org/viewvc/incubator/zeppelin/site/docs/0.6.0-SNAPSHOT/rest-api/rest-configuration.html?rev=1747434&r1=1747433&r2=1747434&view=diff ============================================================================== --- incubator/zeppelin/site/docs/0.6.0-SNAPSHOT/rest-api/rest-configuration.html (original) +++ incubator/zeppelin/site/docs/0.6.0-SNAPSHOT/rest-api/rest-configuration.html Wed Jun 8 19:02:31 2016 @@ -63,6 +63,8 @@
      -
      +
    • Overview
      • +
    • Explore Zeppelin UI
      • +
    • Tutorial
    • Install
      • @@ -71,11 +73,6 @@
    • Upgrade Version
      • - -
    • Tutorial
      • - -
    • UI Layout
      • -
    • Dynamic Form
    • Publish your Paragraph
      • @@ -100,7 +97,7 @@
    • Livy
    • Markdown
    • Python
      • -
    • Postgresql, hawq
      • +
    • Postgresql, HAWQ
    • R
    • Scalding
    • Shell
      • @@ -114,8 +111,8 @@
    • Display System
      • -
      +