kafka-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Gwen Shapira (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (KAFKA-1819) Cleaner gets confused about deleted and re-created topics
Date Tue, 16 Dec 2014 03:08:13 GMT

    [ https://issues.apache.org/jira/browse/KAFKA-1819?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14247661#comment-14247661
] 

Gwen Shapira commented on KAFKA-1819:
-------------------------------------

Yep, looks like a bug.

When we delete a topic we abort the cleanup for the log:
 if (cleaner != null)
        cleaner.abortCleaning(topicAndPartition)

I think we need to add to the LogCleanerManager class a removeLog method. 
This should take a lock, read the checkpoints, remove the log we are deleting from the map
and write the checkpoints back.

Then we need to call it after aborting the cleaning.

If this makes sense to someone who knows what they are doing ([~nehanarkhede] ?) I can implement
this.

> Cleaner gets confused about deleted and re-created topics
> ---------------------------------------------------------
>
>                 Key: KAFKA-1819
>                 URL: https://issues.apache.org/jira/browse/KAFKA-1819
>             Project: Kafka
>          Issue Type: Bug
>            Reporter: Gian Merlino
>            Priority: Blocker
>             Fix For: 0.8.2
>
>
> I get an error like this after deleting a compacted topic and re-creating it. I think
it's because the brokers don't remove cleaning checkpoints from the cleaner-offset-checkpoint
file. This is from a build based off commit bd212b7.
> java.lang.IllegalArgumentException: requirement failed: Last clean offset is 587607 but
segment base offset is 0 for log foo-6.
>         at scala.Predef$.require(Predef.scala:233)
>         at kafka.log.Cleaner.buildOffsetMap(LogCleaner.scala:502)
>         at kafka.log.Cleaner.clean(LogCleaner.scala:300)
>         at kafka.log.LogCleaner$CleanerThread.cleanOrSleep(LogCleaner.scala:214)
>         at kafka.log.LogCleaner$CleanerThread.doWork(LogCleaner.scala:192)
>         at kafka.utils.ShutdownableThread.run(ShutdownableThread.scala:60)



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message