flink-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Triones Deng (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (FLINK-9303) Unassign partitions from Kafka client if partitions become unavailable
Date Mon, 07 May 2018 13:47:00 GMT

    [ https://issues.apache.org/jira/browse/FLINK-9303?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16465923#comment-16465923
] 

Triones Deng commented on FLINK-9303:
-------------------------------------

[~tzulitai] i am interested in this issue. in you jira. there are some conditions will cause
the partitions  become unavailable.
1. the partition leader change will cause the partition unavailable shortly.
2. the topic is deleted as the ML memtioned
3. the partition become offline. like all replication down. 

here what's your idea?

> Unassign partitions from Kafka client if partitions become unavailable
> ----------------------------------------------------------------------
>
>                 Key: FLINK-9303
>                 URL: https://issues.apache.org/jira/browse/FLINK-9303
>             Project: Flink
>          Issue Type: Improvement
>          Components: Kafka Connector
>            Reporter: Tzu-Li (Gordon) Tai
>            Priority: Major
>             Fix For: 1.6.0
>
>
> Originally reported in ML:
> [http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/Dynamically-deleting-kafka-topics-does-not-remove-partitions-from-kafkaConsumer-td19946.html]
> The problem is that the Kafka consumer has no notion of "closed" partitions at the moment,
so statically assigned partitions to the Kafka client is never removed and is always continuously
requested for records.
> This causes LOG noises as reported in the reported mail thread.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Mime
View raw message