ignite-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Nikolay Izhikov (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (IGNITE-3084) Spark Data Frames Support in Apache Ignite
Date Fri, 29 Dec 2017 08:22:00 GMT

    [ https://issues.apache.org/jira/browse/IGNITE-3084?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16306076#comment-16306076
] 

Nikolay Izhikov commented on IGNITE-3084:
-----------------------------------------

[~vkulichenko]

{quote}IGNITE-7337 However, I would really like to have it in 2.4 as well. Do you think that
would be possible?{quote}

As far as I know, code freeze date is January 5.

I think 1 week for implementation, tests, and review is not enough.
Guess we will have some issues with data type converting - from Spark supported to Ignite
SQL supported.

Also, IGNITE-7077(custom strategy and query optimization) is done on 85-90% so I, personally,
prefer to implement IGNITE-7337 after IGNITE-7077 is done.

Val, *IF* we can move code freeze to january, 15 and you have enough time to review my implementation
on january, 5 or similar - I can try to implement these ticket in 2.4

> Spark Data Frames Support in Apache Ignite
> ------------------------------------------
>
>                 Key: IGNITE-3084
>                 URL: https://issues.apache.org/jira/browse/IGNITE-3084
>             Project: Ignite
>          Issue Type: New Feature
>          Components: spark
>    Affects Versions: 1.5.0.final
>            Reporter: Vladimir Ozerov
>            Assignee: Nikolay Izhikov
>            Priority: Critical
>              Labels: bigdata, important
>             Fix For: 2.4
>
>
> Apache Spark already benefits from integration with Apache Ignite. The latter provides
shared RDDs, an implementation of Spark RDD, that help Spark to share a state between Spark
workers and execute SQL queries much faster. The next logical step is to enable support for
modern Spark Data Frames API in a similar way.
> As a contributor, you will be fully in charge of the integration of Spark Data Frame
API and Apache Ignite.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Mime
View raw message