Return-Path: X-Original-To: apmail-zeppelin-commits-archive@minotaur.apache.org Delivered-To: apmail-zeppelin-commits-archive@minotaur.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 3924A18480 for ; Sat, 7 Nov 2015 00:39:24 +0000 (UTC) Received: (qmail 64232 invoked by uid 500); 7 Nov 2015 00:39:24 -0000 Delivered-To: apmail-zeppelin-commits-archive@zeppelin.apache.org Received: (qmail 64186 invoked by uid 500); 7 Nov 2015 00:39:24 -0000 Mailing-List: contact commits-help@zeppelin.incubator.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@zeppelin.incubator.apache.org Delivered-To: mailing list commits@zeppelin.incubator.apache.org Received: (qmail 64167 invoked by uid 99); 7 Nov 2015 00:39:24 -0000 Received: from Unknown (HELO spamd3-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Sat, 07 Nov 2015 00:39:24 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd3-us-west.apache.org (ASF Mail Server at spamd3-us-west.apache.org) with ESMTP id AAF1F1809BB for ; Sat, 7 Nov 2015 00:39:23 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd3-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: 0.791 X-Spam-Level: X-Spam-Status: No, score=0.791 tagged_above=-999 required=6.31 tests=[KAM_ASCII_DIVIDERS=0.8, T_RP_MATCHES_RCVD=-0.01, URIBL_BLOCKED=0.001] autolearn=disabled Received: from mx1-eu-west.apache.org ([10.40.0.8]) by localhost (spamd3-us-west.apache.org [10.40.0.10]) (amavisd-new, port 10024) with ESMTP id m3dyiBJp07qD for ; Sat, 7 Nov 2015 00:39:09 +0000 (UTC) Received: from mailrelay1-us-west.apache.org (mailrelay1-us-west.apache.org [209.188.14.139]) by mx1-eu-west.apache.org (ASF Mail Server at mx1-eu-west.apache.org) with ESMTP id 04E6D20ED5 for ; Sat, 7 Nov 2015 00:39:09 +0000 (UTC) Received: from svn01-us-west.apache.org (svn.apache.org [10.41.0.6]) by mailrelay1-us-west.apache.org (ASF Mail Server at mailrelay1-us-west.apache.org) with ESMTP id B1BA2E00B0 for ; Sat, 7 Nov 2015 00:39:07 +0000 (UTC) Received: from svn01-us-west.apache.org (localhost [127.0.0.1]) by svn01-us-west.apache.org (ASF Mail Server at svn01-us-west.apache.org) with ESMTP id 8FE283A00E5 for ; Sat, 7 Nov 2015 00:39:07 +0000 (UTC) Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Subject: svn commit: r1713055 - in /incubator/zeppelin/site: assets/themes/zeppelin/img/docs-img/ docs/ docs/interpreter/ Date: Sat, 07 Nov 2015 00:39:07 -0000 To: commits@zeppelin.incubator.apache.org From: moon@apache.org X-Mailer: svnmailer-1.0.9 Message-Id: <20151107003907.8FE283A00E5@svn01-us-west.apache.org> Author: moon Date: Sat Nov 7 00:39:07 2015 New Revision: 1713055 URL: http://svn.apache.org/viewvc?rev=1713055&view=rev Log: ZEPPELIN-388 Add documentation for Ignite Interpreter Added: incubator/zeppelin/site/assets/themes/zeppelin/img/docs-img/ignite-interpreter-binding.png (with props) incubator/zeppelin/site/assets/themes/zeppelin/img/docs-img/ignite-interpreter-setting.png (with props) incubator/zeppelin/site/assets/themes/zeppelin/img/docs-img/ignite-logo.png (with props) incubator/zeppelin/site/assets/themes/zeppelin/img/docs-img/ignite-scala-example.png (with props) incubator/zeppelin/site/assets/themes/zeppelin/img/docs-img/ignite-sql-example.png (with props) incubator/zeppelin/site/docs/interpreter/ignite.html Modified: incubator/zeppelin/site/docs/index.html Added: incubator/zeppelin/site/assets/themes/zeppelin/img/docs-img/ignite-interpreter-binding.png URL: http://svn.apache.org/viewvc/incubator/zeppelin/site/assets/themes/zeppelin/img/docs-img/ignite-interpreter-binding.png?rev=1713055&view=auto ============================================================================== Binary file - no diff available. Propchange: incubator/zeppelin/site/assets/themes/zeppelin/img/docs-img/ignite-interpreter-binding.png ------------------------------------------------------------------------------ svn:mime-type = application/octet-stream Added: incubator/zeppelin/site/assets/themes/zeppelin/img/docs-img/ignite-interpreter-setting.png URL: http://svn.apache.org/viewvc/incubator/zeppelin/site/assets/themes/zeppelin/img/docs-img/ignite-interpreter-setting.png?rev=1713055&view=auto ============================================================================== Binary file - no diff available. Propchange: incubator/zeppelin/site/assets/themes/zeppelin/img/docs-img/ignite-interpreter-setting.png ------------------------------------------------------------------------------ svn:mime-type = application/octet-stream Added: incubator/zeppelin/site/assets/themes/zeppelin/img/docs-img/ignite-logo.png URL: http://svn.apache.org/viewvc/incubator/zeppelin/site/assets/themes/zeppelin/img/docs-img/ignite-logo.png?rev=1713055&view=auto ============================================================================== Binary file - no diff available. Propchange: incubator/zeppelin/site/assets/themes/zeppelin/img/docs-img/ignite-logo.png ------------------------------------------------------------------------------ svn:mime-type = application/octet-stream Added: incubator/zeppelin/site/assets/themes/zeppelin/img/docs-img/ignite-scala-example.png URL: http://svn.apache.org/viewvc/incubator/zeppelin/site/assets/themes/zeppelin/img/docs-img/ignite-scala-example.png?rev=1713055&view=auto ============================================================================== Binary file - no diff available. Propchange: incubator/zeppelin/site/assets/themes/zeppelin/img/docs-img/ignite-scala-example.png ------------------------------------------------------------------------------ svn:mime-type = application/octet-stream Added: incubator/zeppelin/site/assets/themes/zeppelin/img/docs-img/ignite-sql-example.png URL: http://svn.apache.org/viewvc/incubator/zeppelin/site/assets/themes/zeppelin/img/docs-img/ignite-sql-example.png?rev=1713055&view=auto ============================================================================== Binary file - no diff available. Propchange: incubator/zeppelin/site/assets/themes/zeppelin/img/docs-img/ignite-sql-example.png ------------------------------------------------------------------------------ svn:mime-type = application/octet-stream Modified: incubator/zeppelin/site/docs/index.html URL: http://svn.apache.org/viewvc/incubator/zeppelin/site/docs/index.html?rev=1713055&r1=1713054&r2=1713055&view=diff ============================================================================== --- incubator/zeppelin/site/docs/index.html (original) +++ incubator/zeppelin/site/docs/index.html Sat Nov 7 00:39:07 2015 @@ -25,7 +25,11 @@ - --> + --> + + + + @@ -172,6 +176,14 @@ + + + + + + + + @@ -289,6 +301,14 @@ + + + + + + + +
  • Download
    • @@ -361,7 +381,7 @@
    • flink
    • hive
      • -
    • ignite
      • +
    • ignite
    • lens
    • md
    • sh
      • @@ -429,9 +449,6 @@ - - - Added: incubator/zeppelin/site/docs/interpreter/ignite.html URL: http://svn.apache.org/viewvc/incubator/zeppelin/site/docs/interpreter/ignite.html?rev=1713055&view=auto ============================================================================== --- incubator/zeppelin/site/docs/interpreter/ignite.html (added) +++ incubator/zeppelin/site/docs/interpreter/ignite.html Sat Nov 7 00:39:07 2015 @@ -0,0 +1,502 @@ + + + + + + Ignite Interpreter + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
      + + + +
      +
      +

      Ignite Interpreter for Apache Zeppelin

      + +

      Overview

      + +

      Apache Ignite In-Memory Data Fabric is a high-performance, integrated and distributed in-memory platform for computing and transacting on large-scale data sets in real-time, orders of magnitude faster than possible with traditional disk-based or flash technologies.

      + +

      Apache Ignite

      + +

      You can use Zeppelin to retrieve distributed data from cache using Ignite SQL interpreter. Moreover, Ignite interpreter allows you to execute any Scala code in cases when SQL doesn't fit to your requirements. For example, you can populate data into your caches or execute distributed computations.

      + +

      Installing and Running Ignite example

      + +

      In order to use Ignite interpreters, you may install Apache Ignite in some simple steps:

      + +
        +

      1. Download Ignite source release or binary release whatever you want. But you must download Ignite as the same version of Zeppelin's. If it is not, you can't use scala code on Zeppelin. You can find ignite version in Zepplin at the pom.xml which is placed under path/to/your-Zeppelin/ignite/pom.xml ( Of course, in Zeppelin source release ). Please check ignite.version .
        Currently, Zeppelin provides ignite only in Zeppelin source release. So, if you download Zeppelin binary release( zeppelin-0.5.0-incubating-bin-spark-xxx-hadoop-xx ), you can not use ignite interpreter on Zeppelin. We are planning to include ignite in a future binary release.

        2. +

      2. Examples are shipped as a separate Maven project, so to start running you simply need to import provided /apache-ignite-fabric-1.2.0-incubating-bin/pom.xml file into your favourite IDE, such as Eclipse.

        3. +
      + +
        +
      • In case of Eclipse, Eclipse -> File -> Import -> Existing Maven Projects
        • +
      • Set examples directory path to Eclipse and select the pom.xml.
        • +

      • Then start org.apache.ignite.examples.ExampleNodeStartup (or whatever you want) to run at least one or more ignite node. When you run example code, you may notice that the number of node is increase one by one.

        + +
        +

        Tip. If you want to run Ignite examples on the cli not IDE, you can export executable Jar file from IDE. Then run it by using below command.

        +
        +
        $ nohup java -jar </path/to/your Jar file name> 
+
        • +
      + +

      Configuring Ignite Interpreter

      + +

      At the "Interpreters" menu, you may edit Ignite interpreter or create new one. Zeppelin provides these properties for Ignite.

      + +

      + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
      Property NamevalueDescription
      ignite.addresses127.0.0.1:47500..47509Coma separated list of Ignite cluster hosts. See Ignite Cluster Configuration section for more details.
      ignite.clientModetrueYou can connect to the Ignite cluster as client or server node. See Ignite Clients vs. Servers section for details. Use true or false values in order to connect in client or server mode respectively.
      ignite.config.urlConfiguration URL. Overrides all other settings.
      ignite.jdbc.urljdbc:ignite:cfg://default-ignite-jdbc.xmlIgnite JDBC connection URL.
      ignite.peerClassLoadingEnabledtrueEnables peer-class-loading. See Zero Deployment section for details. Use true or false values in order to enable or disable P2P class loading respectively.

      + +

      Configuration of Ignite Interpreter

      + +

      Interpreter Binding for Zeppelin Notebook

      + +

      After configuring Ignite interpreter, create your own notebook. Then you can bind interpreters like below image.

      + +

      Binding Interpreters

      + +

      For more interpreter binding information see here.

      + +

      How to use Ignite SQL interpreter

      + +

      In order to execute SQL query, use %ignite.ignitesql prefix.
      +Supposing you are running org.apache.ignite.examples.streaming.wordcount.StreamWords, then you can use "words" cache( Of course you have to specify this cache name to the Ignite interpreter setting section ignite.jdbc.url of Zeppelin ). +For example, you can select top 10 words in the words cache using the following query

      +
        %ignite.ignitesql 
+  select _val, count(_val) as cnt from String group by _val order by cnt desc limit 10 
+
      +

      IgniteSql on Zeppelin

      + +

      As long as your Ignite version and Zeppelin Ignite version is same, you can also use scala code. Please check the Zeppelin Ignite version before you download your own Ignite.

      +
        %ignite
+  import org.apache.ignite._
+  import org.apache.ignite.cache.affinity._
+  import org.apache.ignite.cache.query._
+  import org.apache.ignite.configuration._
+
+  import scala.collection.JavaConversions._
+
+  val cache: IgniteCache[AffinityUuid, String] = ignite.cache("words")
+
+  val qry = new SqlFieldsQuery("select avg(cnt), min(cnt), max(cnt) from (select count(_val) as cnt from String group by _val)", true)
+
+  val res = cache.query(qry).getAll()
+
+  collectionAsScalaIterable(res).foreach(println _)
+
      +

      Using Scala Code

      + +

      Apache Ignite also provides a guide docs for Zeppelin "Ignite with Apache Zeppelin"

      + +
      +
      + + +
      +
      + +
      +
      + + + + + + + + + + +