flink-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Konstantin Lalafaryan (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (FLINK-7913) Add support for Kafka default partitioner
Date Mon, 11 Dec 2017 11:54:01 GMT

    [ https://issues.apache.org/jira/browse/FLINK-7913?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16285816#comment-16285816

Konstantin Lalafaryan commented on FLINK-7913:

- Kafka uses round-robin partitioner
- flink uses parallel instance id to decide the target partition. So let's say you have 12
kafka partitions and only two flink partitions in this case only two kafka partitions will
be used. You can check the logic here: org.apache.flink.streaming.connectors.kafka.partitioner.FixedPartitioner

> Add support for Kafka default partitioner
> -----------------------------------------
>                 Key: FLINK-7913
>                 URL: https://issues.apache.org/jira/browse/FLINK-7913
>             Project: Flink
>          Issue Type: Improvement
>          Components: Kafka Connector
>    Affects Versions: 1.4.0
>            Reporter: Konstantin Lalafaryan
>            Assignee: Konstantin Lalafaryan
>             Fix For: 1.5.0
> Currently in the Apache Flink it is available only *FlinkKafkaPartitioner* and just one
implementation *FlinkFixedPartitioner*. 
> In order to be able to use Kafka's default partitioner you have to create new implementation
for *FlinkKafkaPartitioner* and fork the code from the Kafka. It will be really good to be
able to define the partitioner without implementing the new class.
> Thanks.

This message was sent by Atlassian JIRA

View raw message