kafka-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Vahid Hashemian (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (KAFKA-3370) Add options to auto.offset.reset to reset offsets upon initialization only
Date Wed, 25 May 2016 00:40:13 GMT

    [ https://issues.apache.org/jira/browse/KAFKA-3370?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15299249#comment-15299249

Vahid Hashemian commented on KAFKA-3370:

[~gwenshap] Thanks for your feedback, If I understand you correctly, you are describing an
offset reset policy that is "smart" (for lack of a better term), and acts as "earliest" or
"latest" depending on the situation to improve efficiency.

> Add options to auto.offset.reset to reset offsets upon initialization only
> --------------------------------------------------------------------------
>                 Key: KAFKA-3370
>                 URL: https://issues.apache.org/jira/browse/KAFKA-3370
>             Project: Kafka
>          Issue Type: Bug
>            Reporter: Guozhang Wang
>            Assignee: Vahid Hashemian
>             Fix For:
> Currently "auto.offset.reset" is applied in the following two cases:
> 1) upon starting the consumer for the first time (hence no committed offsets before);
> 2) upon fetching offsets out-of-range.
> For scenarios where case 2) needs to be avoid (i.e. people need to be notified upon offsets
out-of-range rather than silently offset reset), "auto.offset.reset" need to be set to "none".
However for case 1) setting "auto.offset.reset" to "none" will cause NoOffsetForPartitionException
upon polling. And in this case, seekToBeginning/seekToEnd is mistakenly applied trying to
set the offset at initialization, which are actually designed for during the life time of
the consumer (in rebalance callback, for example).
> The fix proposal is to add two more options to "auto.offset.reset", "earliest-on-start",
and "latest-on-start", whose semantics are "earliest" and "latest" for case 1) only, and "none"
for case 2).

This message was sent by Atlassian JIRA

View raw message