pig-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Pradeep Gollakota (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (PIG-3453) Implement a Storm backend to Pig
Date Mon, 04 Nov 2013 19:22:19 GMT

    [ https://issues.apache.org/jira/browse/PIG-3453?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13813136#comment-13813136

Pradeep Gollakota commented on PIG-3453:

[~thedatachef] Wow... This is a great start! Thanks so much for hacking this out.

[~cheolsoo] I agree that this work should be committed back to a branch on Apache. I foresee
a lot more contributions and collaboration on this, so it would be easier to coordinate via
Apache as opposed to the git mirror.

[~dvryaboy] I have been strongly considering writing this DSL to Summingbird instead of Trident/Storm.
I am considering if there are going to be any implications to doing this though. By writing
to Summingbird we would get both a real-time mode and a hybrid mode execution, which in my
mind is a huge win. At Lithium Technologies, we have been considering using Summingbird for
hybrid mode execution. The question in my mind is, do we want to use Summingbird if all we
want is a real-time engine (i.e. storm). We can change the scope of this JIRA to write a Summingbird
backend or we can open another JIRA to implement a Summingbird POC and then see where that
gets us.

> Implement a Storm backend to Pig
> --------------------------------
>                 Key: PIG-3453
>                 URL: https://issues.apache.org/jira/browse/PIG-3453
>             Project: Pig
>          Issue Type: New Feature
>    Affects Versions: 0.13.0
>            Reporter: Pradeep Gollakota
>            Assignee: Jacob Perkins
>              Labels: storm
>             Fix For: 0.13.0
>         Attachments: storm-integration.patch
> There is a lot of interest around implementing a Storm backend to Pig for streaming processing.
The proposal and initial discussions can be found at https://cwiki.apache.org/confluence/display/PIG/Pig+on+Storm+Proposal

This message was sent by Atlassian JIRA

View raw message