crunch-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Micah Whitacre (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (CRUNCH-606) Create a KafkaSource
Date Mon, 16 May 2016 15:45:13 GMT

     [ https://issues.apache.org/jira/browse/CRUNCH-606?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Micah Whitacre updated CRUNCH-606:
----------------------------------
    Attachment: CRUNCH-606b.diff

Got some code review feedback from my team and here's the updated commits:
https://github.com/mkwhitacre/crunch/commits/CRUNCH-606-byteswritable

Here's the changes as a diff as well.  I'll squash and give prettier commit message before
pushing to master.




> Create a KafkaSource
> --------------------
>
>                 Key: CRUNCH-606
>                 URL: https://issues.apache.org/jira/browse/CRUNCH-606
>             Project: Crunch
>          Issue Type: New Feature
>          Components: IO
>            Reporter: Micah Whitacre
>            Assignee: Micah Whitacre
>         Attachments: CRUNCH-606-byteswritable-singleuse.diff, CRUNCH-606-byteswritable.diff,
CRUNCH-606.diff, CRUNCH-606.patch, CRUNCH-606b.diff
>
>
> Pulling data out of Kafka is a common use case and some of the ways to do it Kafka Connect,
Camus, Gobblin do not integrate nicely with existing processing pipelines like Crunch.  With
Kafka 0.9, the consuming API is a lot easier so we should build a Source implementation that
can read from Kafka.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message