ignite-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Denis Magda (Jira)" <j...@apache.org>
Subject [jira] [Commented] (IGNITE-12054) Upgrade Spark module to 2.4
Date Wed, 11 Sep 2019 21:19:00 GMT

    [ https://issues.apache.org/jira/browse/IGNITE-12054?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16928005#comment-16928005
] 

Denis Magda commented on IGNITE-12054:
--------------------------------------

[~NIzhikov], [~zaleslaw], how about creating the first separate Ignite module for Spark and
release it independently? The module will be in its own repository and there are might be
different branches for different versions. Ignite community will support and release only
the latest version of Spark integration while the users can always build an old version from
one of the branches.

More details are recorded on this page:
https://cwiki.apache.org/confluence/display/IGNITE/IEP-36%3A+Modularization

If three of us agree, then we can restart discussion on the dev list, do a call with [~agoncharuk]
and other community members and get down to the implementation. 

> Upgrade Spark module to 2.4
> ---------------------------
>
>                 Key: IGNITE-12054
>                 URL: https://issues.apache.org/jira/browse/IGNITE-12054
>             Project: Ignite
>          Issue Type: Task
>          Components: spark
>    Affects Versions: 2.7.5
>            Reporter: Denis Magda
>            Assignee: Nikolay Izhikov
>            Priority: Blocker
>             Fix For: 2.8
>
>
> Users can't use APIs that are already available in Spark 2.4:
> https://stackoverflow.com/questions/57392143/persisting-spark-dataframe-to-ignite
> Let's upgrade Spark from 2.3 to 2.4 until we extract the Spark Integration as a separate
module that can support multiple Spark versions.



--
This message was sent by Atlassian Jira
(v8.3.2#803003)

Mime
View raw message