beam-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF GitHub Bot (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (BEAM-2534) KafkaIO should allow gaps in message offsets
Date Mon, 10 Jul 2017 19:55:00 GMT

    [ https://issues.apache.org/jira/browse/BEAM-2534?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16080984#comment-16080984
] 

ASF GitHub Bot commented on BEAM-2534:
--------------------------------------

Github user asfgit closed the pull request at:

    https://github.com/apache/beam/pull/3514


> KafkaIO should allow gaps in message offsets
> --------------------------------------------
>
>                 Key: BEAM-2534
>                 URL: https://issues.apache.org/jira/browse/BEAM-2534
>             Project: Beam
>          Issue Type: Bug
>          Components: sdk-java-extensions
>    Affects Versions: 2.0.0
>            Reporter: Raghu Angadi
>            Assignee: Raghu Angadi
>            Priority: Minor
>             Fix For: 2.1.0
>
>
> KafkaIO reader logs a warning when it notices gaps in offsets for messages. While such
gaps are not expected for normal Kafka topics, there could be gaps when log compaction is
enabled (which deletes older messages for a key). 
> This warning log is not very useful. Also we should take such gaps while estimating backlog.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Mime
View raw message