flink-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF GitHub Bot (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (FLINK-9303) Unassign partitions from Kafka client if partitions become unavailable
Date Wed, 16 May 2018 09:44:00 GMT

    [ https://issues.apache.org/jira/browse/FLINK-9303?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16477171#comment-16477171
] 

ASF GitHub Bot commented on FLINK-9303:
---------------------------------------

Github user EAlexRojas commented on a diff in the pull request:

    https://github.com/apache/flink/pull/5991#discussion_r188564095
  
    --- Diff: flink-connectors/flink-connector-kafka-0.8/src/main/java/org/apache/flink/streaming/connectors/kafka/FlinkKafkaConsumer08.java
---
    @@ -221,7 +221,8 @@ private FlinkKafkaConsumer08(
     				getLong(
     					checkNotNull(props, "props"),
     					KEY_PARTITION_DISCOVERY_INTERVAL_MILLIS, PARTITION_DISCOVERY_DISABLED),
    -				!getBoolean(props, KEY_DISABLE_METRICS, false));
    +				!getBoolean(props, KEY_DISABLE_METRICS, false),
    +				getBoolean(props, KEY_CHECK_UNAVAILABLE_TOPICS, false));
    --- End diff --
    
    You're right, I'll change it


> Unassign partitions from Kafka client if partitions become unavailable
> ----------------------------------------------------------------------
>
>                 Key: FLINK-9303
>                 URL: https://issues.apache.org/jira/browse/FLINK-9303
>             Project: Flink
>          Issue Type: Improvement
>          Components: Kafka Connector
>            Reporter: Tzu-Li (Gordon) Tai
>            Priority: Major
>             Fix For: 1.6.0
>
>
> Originally reported in ML:
> [http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/Dynamically-deleting-kafka-topics-does-not-remove-partitions-from-kafkaConsumer-td19946.html]
> The problem is that the Kafka consumer has no notion of "closed" partitions at the moment,
so statically assigned partitions to the Kafka client is never removed and is always continuously
requested for records.
> This causes LOG noises as reported in the reported mail thread.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Mime
View raw message