datafu-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Matthew Hayes (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (DATAFU-148) Setup Spark sub-project
Date Thu, 11 Oct 2018 01:10:00 GMT

    [ https://issues.apache.org/jira/browse/DATAFU-148?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16645798#comment-16645798
] 

Matthew Hayes commented on DATAFU-148:
--------------------------------------

Great! I'll take a look at the changes you pushed.

 flatten and changeSchema seem useful.  What do you think?  I'd add docs and tests if you
agree.

Regarding the versions, let's check what others are doing.  My inclination would be to start
with the most recent release and add more as there is demand.  Check the Apache Samza project
for Gradle examples.  They use Scala I believe and Gradle so they probably have a way to
deal with multiple versions.

> Setup Spark sub-project
> -----------------------
>
>                 Key: DATAFU-148
>                 URL: https://issues.apache.org/jira/browse/DATAFU-148
>             Project: DataFu
>          Issue Type: New Feature
>            Reporter: Eyal Allweil
>            Assignee: Eyal Allweil
>            Priority: Major
>
> Create a skeleton Spark sub project for Spark code to be contributed to DataFu



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Mime
View raw message