flink-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Robert Metzger (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (FLINK-2386) Implement Kafka connector using the new Kafka Consumer API
Date Tue, 21 Jul 2015 15:24:04 GMT

    [ https://issues.apache.org/jira/browse/FLINK-2386?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14635243#comment-14635243
] 

Robert Metzger commented on FLINK-2386:
---------------------------------------

My current work in progress code is located here: https://github.com/rmetzger/flink/tree/flink2386
It contains a full copy of the new kafka consumer code (because it is not yet released).
I'll probably finish the changes once Kafka has released the new consumer.

> Implement Kafka connector using the new Kafka Consumer API
> ----------------------------------------------------------
>
>                 Key: FLINK-2386
>                 URL: https://issues.apache.org/jira/browse/FLINK-2386
>             Project: Flink
>          Issue Type: Improvement
>          Components: Kafka Connector
>            Reporter: Robert Metzger
>
> Once Kafka has released its new consumer API, we should provide a connector for that
version.
> The release will probably be called 0.9 or 0.8.3.
> The connector will be mostly compatible with Kafka 0.8.2.x, except for committing offsets
to the broker (the new connector expects a coordinator to be available on Kafka). To work
around that, we can provide a configuration option to commit offsets to zookeeper (managed
by flink code).
> For 0.9/0.8.3 it will be fully compatible.
> It will not be compatible with 0.8.1 because of mismatching Kafka messages.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message