datafu-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Eyal Allweil (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (DATAFU-148) Setup Spark sub-project
Date Tue, 11 Jun 2019 09:54:00 GMT

    [ https://issues.apache.org/jira/browse/DATAFU-148?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16860761#comment-16860761
] 

Eyal Allweil commented on DATAFU-148:
-------------------------------------

There's indeed a new version of spark-testing-base which supports Scala 2.12, so I pushed
a fix that uses that and supports 2.12 for Spark 2.4.0. 2.4.1 and newer made a change in how
they use the py4j gateway which will require an additional fix (for Scala 2.11 as well, not
just for 2.12). I'm looking into it.

> Setup Spark sub-project
> -----------------------
>
>                 Key: DATAFU-148
>                 URL: https://issues.apache.org/jira/browse/DATAFU-148
>             Project: DataFu
>          Issue Type: New Feature
>            Reporter: Eyal Allweil
>            Assignee: Eyal Allweil
>            Priority: Major
>         Attachments: patch.diff, patch.diff
>
>          Time Spent: 40m
>  Remaining Estimate: 0h
>
> Create a skeleton Spark sub project for Spark code to be contributed to DataFu



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Mime
View raw message