ignite-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Manish Mishra (JIRA)" <j...@apache.org>
Subject [jira] [Comment Edited] (IGNITE-4526) Add Spark Shared RDD examples
Date Sun, 08 Jan 2017 16:44:58 GMT

    [ https://issues.apache.org/jira/browse/IGNITE-4526?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15809663#comment-15809663
] 

Manish Mishra edited comment on IGNITE-4526 at 1/8/17 4:44 PM:
---------------------------------------------------------------

The Above PR contains a very basic example of instantiating the IgniteRDD and filling values
into it. The above PR was necessary to check if the build is passing for the added code so
far, and for test-running the Teamcity. I will continue to add examples w.r.t above mentioned
task description. [~dmagda] Does it look ok ? I need some feedback on current work. 


was (Author: manish__mishra):
The Above PR contains a very basic example of instantiating the IgniteRDD and filling values
into it. The above PR was necessary to check if the build is passing for the added code so
far, and for test-running the Teamcity. I will continue to add examples w.r.t above mentioned
task description. 

> Add Spark Shared RDD examples
> -----------------------------
>
>                 Key: IGNITE-4526
>                 URL: https://issues.apache.org/jira/browse/IGNITE-4526
>             Project: Ignite
>          Issue Type: Task
>            Reporter: Denis Magda
>            Assignee: Manish Mishra
>             Fix For: 2.0
>
>
> Spark Shared RDD functionality doesn't have its own examples. We need to add an example
that will do the following:
> - First Spark Worker: creation of a shared RDD and filling it in with data.
> - First Spark Worker: performing some native spark transformation with the RDD.
> - Second Spark Worker: connecting to the same shared RDD.
> - Second Spark Worker: execution of SQL query using Spark API and Ignite API. Show that
Ignite's query executes faster.
> The reason why the example should consist of two workers is to showcase one of the main
benefits of Ignite's RDDs - ability to share the state (RDD) amid different Spark workers
and processes.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message