crunch-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Josh Wills (JIRA)" <>
Subject [jira] [Updated] (CRUNCH-496) Scrunch on Spark
Date Sat, 31 Jan 2015 17:03:35 GMT


Josh Wills updated CRUNCH-496:
    Attachment: CRUNCH-496.patch

I got Scrunch running on top of the SparkPipeline last night after making a small hack to
the Spark closure serializer so that it would properly load serialized primitive types (float,
int, byte, etc.), which weren't working for some reason.

Will likely iterate on this one a bit more, but feedback is welcome.

> Scrunch on Spark
> ----------------
>                 Key: CRUNCH-496
>                 URL:
>             Project: Crunch
>          Issue Type: New Feature
>          Components: Scrunch
>            Reporter: Josh Wills
>         Attachments: CRUNCH-496.patch
> Add an implementation of Scrunch (specifically, a subclass of o.a.c.scrunch.Pipeline)
that wraps SparkPipeline.

This message was sent by Atlassian JIRA

View raw message