kafka-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF GitHub Bot (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (KAFKA-4529) tombstone may be removed earlier than it should
Date Thu, 15 Dec 2016 05:55:58 GMT

    [ https://issues.apache.org/jira/browse/KAFKA-4529?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15750478#comment-15750478

ASF GitHub Bot commented on KAFKA-4529:

GitHub user becketqin opened a pull request:


    KAFKA-4529; LogCleaner should not delete the tombstone too early.

    cc @junrao 

You can merge this pull request into a Git repository by running:

    $ git pull https://github.com/becketqin/kafka KAFKA-4529-trunk

Alternatively you can review and apply these changes as the patch at:


To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

    This closes #2260
commit 681613bb7acd2df2a3abd1519dab1272a5b3c207
Author: Jiangjie Qin <becket.qin@gmail.com>
Date:   2016-12-15T05:53:35Z

    KAFKA-4529; LogCleaner should not delete the tombstone too early.


> tombstone may be removed earlier than it should
> -----------------------------------------------
>                 Key: KAFKA-4529
>                 URL: https://issues.apache.org/jira/browse/KAFKA-4529
>             Project: Kafka
>          Issue Type: Bug
>    Affects Versions:
>            Reporter: Jun Rao
>            Assignee: Jiangjie Qin
>             Fix For:
> As part of KIP-33, we introduced a regression on how tombstone is removed in a compacted
topic. We want to delay the removal of a tombstone to avoid the case that a reader first reads
a non-tombstone message on a key and then doesn't see the tombstone for the key because it's
deleted too quickly. So, a tombstone is supposed to only be removed from a compacted topic
after the tombstone is part of the cleaned portion of the log after delete.retention.ms.
> Before KIP-33, deleteHorizonMs in LogCleaner is calculated based on the last modified
time, which is monotonically increasing from old to new segments. With KIP-33, deleteHorizonMs
is calculated based on the message timestamp, which is not necessarily monotonically increasing.

This message was sent by Atlassian JIRA

View raw message