kafka-jira mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jason Gustafson (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (KAFKA-6492) LogSemgent.truncateTo() should always resize the index file
Date Wed, 31 Jan 2018 18:59:00 GMT

    [ https://issues.apache.org/jira/browse/KAFKA-6492?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16347394#comment-16347394

Jason Gustafson commented on KAFKA-6492:

[~becket_qin] Does this only affect 1.0.0? As far as I can tell, it can happen on older versions
as well. I have seen recently an instance of the failure during log rolling on 0.10.1. It
was a compacted topic, but I cannot confirm whether it is the specific truncation scenario
you describe above (I don't have all the historic logs).

> LogSemgent.truncateTo() should always resize the index file
> -----------------------------------------------------------
>                 Key: KAFKA-6492
>                 URL: https://issues.apache.org/jira/browse/KAFKA-6492
>             Project: Kafka
>          Issue Type: Bug
>          Components: core
>    Affects Versions: 1.0.0
>            Reporter: Jiangjie Qin
>            Priority: Major
>             Fix For: 1.1.0
> The bug is the following:
>  # Initially on a follower broker there are two segments 0 and segment 10000. Segment
0 is empty (maybe due to log compaction)
>  # log is truncated to 0.
>  # LogSemgent.Truncate() will not find a message to truncate in segment 0, so it will
skip resizing the index/timeindex files. 
>  # When a new message is fetched, Log.maybeRoll() will try to roll a new segment because
the index file of segment 0 is already full (max size is 0)
>  # After creating the new segment 0, the replica fetcher thread finds that there is already
a segment 0 exists. So it just throws exception and dies.
> The fix would be let the broker make sure the index files of active segments are always
resized properly.

This message was sent by Atlassian JIRA

View raw message