kafka-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Gwen Shapira (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (KAFKA-1819) Cleaner gets confused about deleted and re-created topics
Date Tue, 16 Dec 2014 18:20:13 GMT

    [ https://issues.apache.org/jira/browse/KAFKA-1819?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14248602#comment-14248602

Gwen Shapira commented on KAFKA-1819:

Thanks for pointing the similar issue [~jjkoshy].

The log is indeed removed from the pool in LogManager.deleteLog, and we could remove them
in doneCleaning. 

However, I think we want to be able to force cleaning as part of the topic delete.
If we don't do it, the checkpoint file will only get updated some time later when doneCleaning
is called. This can be more challenging to troubleshoot and also may not happen before Gian
creates a new topic with same name.

> Cleaner gets confused about deleted and re-created topics
> ---------------------------------------------------------
>                 Key: KAFKA-1819
>                 URL: https://issues.apache.org/jira/browse/KAFKA-1819
>             Project: Kafka
>          Issue Type: Bug
>            Reporter: Gian Merlino
>            Priority: Blocker
>             Fix For: 0.8.2
> I get an error like this after deleting a compacted topic and re-creating it. I think
it's because the brokers don't remove cleaning checkpoints from the cleaner-offset-checkpoint
file. This is from a build based off commit bd212b7.
> java.lang.IllegalArgumentException: requirement failed: Last clean offset is 587607 but
segment base offset is 0 for log foo-6.
>         at scala.Predef$.require(Predef.scala:233)
>         at kafka.log.Cleaner.buildOffsetMap(LogCleaner.scala:502)
>         at kafka.log.Cleaner.clean(LogCleaner.scala:300)
>         at kafka.log.LogCleaner$CleanerThread.cleanOrSleep(LogCleaner.scala:214)
>         at kafka.log.LogCleaner$CleanerThread.doWork(LogCleaner.scala:192)
>         at kafka.utils.ShutdownableThread.run(ShutdownableThread.scala:60)

This message was sent by Atlassian JIRA

View raw message