flink-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF GitHub Bot (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (FLINK-2974) Add periodic offset commit to Kafka Consumer if checkpointing is disabled
Date Fri, 20 Nov 2015 11:54:11 GMT

    [ https://issues.apache.org/jira/browse/FLINK-2974?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15015983#comment-15015983
] 

ASF GitHub Bot commented on FLINK-2974:
---------------------------------------

Github user StephanEwen commented on a diff in the pull request:

    https://github.com/apache/flink/pull/1341#discussion_r45461519
  
    --- Diff: flink-streaming-connectors/flink-connector-kafka/src/main/java/org/apache/flink/streaming/connectors/kafka/FlinkKafkaConsumer.java
---
    @@ -290,9 +291,6 @@ public FlinkKafkaConsumer(String topic, DeserializationSchema<T>
valueDeserializ
     			}
     		}
     		LOG.info("Topic {} has {} partitions", topic, partitions.length);
    -
    -		// make sure that we take care of the committing
    -		props.setProperty("enable.auto.commit", "false");
    --- End diff --
    
    Is this not necessary, because the low-level consumer does not respect it anyways?


> Add periodic offset commit to Kafka Consumer if checkpointing is disabled
> -------------------------------------------------------------------------
>
>                 Key: FLINK-2974
>                 URL: https://issues.apache.org/jira/browse/FLINK-2974
>             Project: Flink
>          Issue Type: Improvement
>          Components: Kafka Connector
>            Reporter: Robert Metzger
>            Assignee: Robert Metzger
>
> Flink only writes the offsets from the consumer into ZK if checkpointing is enabled.
> We should have a similar feature to Kafka's autocommit in our consumer.
> Issue reported by user: http://stackoverflow.com/questions/33501574/flink-kafka-why-am-i-losing-messages



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message