kafka-jira mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Tomáš Králík (JIRA) <j...@apache.org>
Subject [jira] [Commented] (KAFKA-6099) Seeking consumer to evicted offset resets the offset
Date Tue, 13 Feb 2018 09:57:00 GMT

    [ https://issues.apache.org/jira/browse/KAFKA-6099?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16362072#comment-16362072

Tomáš Králík commented on KAFKA-6099:

This is bit old - but for whoever finds this - there is a new option to {code:java}auto.offset.reset{code}
consumer property. When set to "none" or anything else, exception is thrown. See the [docs|https://kafka.apache.org/documentation/#newconsumerconfigs].


> Seeking consumer to evicted offset resets the offset
> ----------------------------------------------------
>                 Key: KAFKA-6099
>                 URL: https://issues.apache.org/jira/browse/KAFKA-6099
>             Project: Kafka
>          Issue Type: Bug
>          Components: consumer
>    Affects Versions:
>         Environment: Windows
>            Reporter: Viliam Durina
>            Priority: Major
> We use manual partition assignment and save the offsets to our storage. The topic also
has short "retention.ms" property. When we try to restart consumption from an already evicted
offset, the offset is reset according to the "auto.offset.reset" property. That is:
> - if "latest" is configured, it only returns records that were inserted after the `seek`
> - if "earliest" is configured, it tries to restart at 0, which fails for the same reason,
because offset=0 is also evicted.
> Expected behavior is to report the situation with an exception, thrown from either `seek()`
or `poll()` call. The user will then be expected to `seekToBeginning` or `seekToEnd` or to
any other position.
> Another option is to restart at earliest available record, regardless of the value of
the "auto.offset.reset" property. However, this way the consumer has no way of knowing that
it missed some records it expected.

This message was sent by Atlassian JIRA

View raw message