pig-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Pradeep Gollakota (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (PIG-3453) Implement a Storm backend to Pig
Date Wed, 11 Sep 2013 21:34:51 GMT

    [ https://issues.apache.org/jira/browse/PIG-3453?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13764812#comment-13764812
] 

Pradeep Gollakota commented on PIG-3453:
----------------------------------------

I personally don't have a concrete use case for this yet. In terms of using a system that
can work both in warehousing and in real-time, I have been looking at Summingbird (recently
opensourced). I think the word count example is a good place to start as it's the canonical
example. However, I'd like to have a more complicated example as well, so I'm writing a TF-IDF
implementation in Pig and in Trident. Perhaps, this can be step 2 PoC after word count. I'd
also like to cut out some of the more complex operations like nested foreach statements etc
in the initial PoC. I'm not sure yet how we'd solve them.

[~thedatachef] I started a new job last week and I'm not sure how this task would fit into
the road map of my new company yet. I'd love to work on this, if I have time. You're more
than welcome to work on this as well. Thanks for all your great comments, input and enthusiasm.
                
> Implement a Storm backend to Pig
> --------------------------------
>
>                 Key: PIG-3453
>                 URL: https://issues.apache.org/jira/browse/PIG-3453
>             Project: Pig
>          Issue Type: New Feature
>            Reporter: Pradeep Gollakota
>              Labels: storm
>
> There is a lot of interest around implementing a Storm backend to Pig for streaming processing.
The proposal and initial discussions can be found at https://cwiki.apache.org/confluence/display/PIG/Pig+on+Storm+Proposal

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message