flink-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Tzu-Li (Gordon) Tai (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (FLINK-3398) Flink Kafka consumer should support auto-commit opt-outs
Date Fri, 29 Jul 2016 16:31:20 GMT

    [ https://issues.apache.org/jira/browse/FLINK-3398?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15399625#comment-15399625

Tzu-Li (Gordon) Tai commented on FLINK-3398:

[~StephanEwen] That's actually a very good summary! From the usage pattern, I think it makes
sense. Our Kafka consumers are using "group.id" for start offset & offset committing only,
so the change will also fit into the current implementation.

However, this again is stretching Kafka's original intent of the "group.id" config. I think
we need to decide whether or not these Flink-specific behaviours should be separated apart
from Kafka's original configs.

> Flink Kafka consumer should support auto-commit opt-outs
> --------------------------------------------------------
>                 Key: FLINK-3398
>                 URL: https://issues.apache.org/jira/browse/FLINK-3398
>             Project: Flink
>          Issue Type: Improvement
>          Components: Kafka Connector
>            Reporter: Shikhar Bhushan
>             Fix For: 1.2.0
> Currently the Kafka source will commit consumer offsets to Zookeeper, either upon a checkpoint
if checkpointing is enabled, otherwise periodically based on {{auto.commit.interval.ms}}
> It should be possible to opt-out of committing consumer offsets to Zookeeper. Kafka has
this config as {{auto.commit.enable}} (0.8) and {{enable.auto.commit}} (0.9).

This message was sent by Atlassian JIRA

View raw message