Return-Path: X-Original-To: apmail-kafka-dev-archive@www.apache.org Delivered-To: apmail-kafka-dev-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 8C8F318D78 for ; Mon, 4 Jan 2016 17:20:40 +0000 (UTC) Received: (qmail 53065 invoked by uid 500); 4 Jan 2016 17:20:40 -0000 Delivered-To: apmail-kafka-dev-archive@kafka.apache.org Received: (qmail 52931 invoked by uid 500); 4 Jan 2016 17:20:40 -0000 Mailing-List: contact dev-help@kafka.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@kafka.apache.org Delivered-To: mailing list dev@kafka.apache.org Received: (qmail 52907 invoked by uid 99); 4 Jan 2016 17:20:40 -0000 Received: from arcas.apache.org (HELO arcas) (140.211.11.28) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 04 Jan 2016 17:20:40 +0000 Received: from arcas.apache.org (localhost [127.0.0.1]) by arcas (Postfix) with ESMTP id F19E12C14F4 for ; Mon, 4 Jan 2016 17:20:39 +0000 (UTC) Date: Mon, 4 Jan 2016 17:20:39 +0000 (UTC) From: "Henri Pihkala (JIRA)" To: dev@kafka.apache.org Message-ID: In-Reply-To: References: Subject: [jira] [Commented] (KAFKA-2937) Topics marked for delete in Zookeeper may become undeletable MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394 [ https://issues.apache.org/jira/browse/KAFKA-2937?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15081401#comment-15081401 ] Henri Pihkala commented on KAFKA-2937: -------------------------------------- [~mgharat] Yes, like this: AdminUtils.createTopic(...) assert AdminUtils.topicExists(...) AdminUtils.deleteTopic(...) Thread.sleep(...) assert !AdminUtils.topicExists(...) When this test one day suddenly started failing (on the last line), I first suspected that my sleep wasn't long enough anymore. However, digging deeper showed that topics created by my test were ending up in the undeleteable state described in this JIRA. > Topics marked for delete in Zookeeper may become undeletable > ------------------------------------------------------------ > > Key: KAFKA-2937 > URL: https://issues.apache.org/jira/browse/KAFKA-2937 > Project: Kafka > Issue Type: Bug > Components: core > Affects Versions: 0.9.0.0 > Reporter: Rajini Sivaram > Assignee: Mayuresh Gharat > > In our clusters, we occasionally see topics marked for delete, but never actually deleted. It may be due to brokers being restarted while tests were running, but further restarts of Kafka dont fix the problem. The topics remain marked for delete in Zookeeper. > Topic describe shows: > {quote} > Topic:testtopic PartitionCount:1 ReplicationFactor:3 Configs: > Topic: testtopic Partition: 0 Leader: none Replicas: 3,4,0 Isr: > {quote} > Kafka logs show: > {quote} > 2015-12-02 15:53:30,152] ERROR Controller 2 epoch 213 initiated state change of replica 3 for partition [testtopic,0] from OnlineReplica to OfflineReplica failed (state.change.logger) > kafka.common.StateChangeFailedException: Failed to change state of replica 3 for partition [testtopic,0] since the leader and isr path in zookeeper is empty > at kafka.controller.ReplicaStateMachine.handleStateChange(ReplicaStateMachine.scala:269) > at kafka.controller.ReplicaStateMachine$$anonfun$handleStateChanges$2.apply(ReplicaStateMachine.scala:114) > at kafka.controller.ReplicaStateMachine$$anonfun$handleStateChanges$2.apply(ReplicaStateMachine.scala:114) > at scala.collection.immutable.HashSet$HashSet1.foreach(HashSet.scala:322) > at scala.collection.immutable.HashSet$HashTrieSet.foreach(HashSet.scala:978) > at kafka.controller.ReplicaStateMachine.handleStateChanges(ReplicaStateMachine.scala:114) > at kafka.controller.TopicDeletionManager$$anonfun$startReplicaDeletion$2.apply(TopicDeletionManager.scala:342) > at kafka.controller.TopicDeletionManager$$anonfun$startReplicaDeletion$2.apply(TopicDeletionManager.scala:334) > at scala.collection.immutable.Map$Map1.foreach(Map.scala:116) > at kafka.controller.TopicDeletionManager.startReplicaDeletion(TopicDeletionManager.scala:334) > at kafka.controller.TopicDeletionManager.kafka$controller$TopicDeletionManager$$onPartitionDeletion(TopicDeletionManager.scala:367) > at kafka.controller.TopicDeletionManager$$anonfun$kafka$controller$TopicDeletionManager$$onTopicDeletion$2.apply(TopicDeletionManager.scala:313) > at kafka.controller.TopicDeletionManager$$anonfun$kafka$controller$TopicDeletionManager$$onTopicDeletion$2.apply(TopicDeletionManager.scala:312) > at scala.collection.immutable.Set$Set1.foreach(Set.scala:79) > at kafka.controller.TopicDeletionManager.kafka$controller$TopicDeletionManager$$onTopicDeletion(TopicDeletionManager.scala:312) > at kafka.controller.TopicDeletionManager$DeleteTopicsThread$$anonfun$doWork$1$$anonfun$apply$mcV$sp$4.apply(TopicDeletionManager.scala:431) > at kafka.controller.TopicDeletionManager$DeleteTopicsThread$$anonfun$doWork$1$$anonfun$apply$mcV$sp$4.apply(TopicDeletionManager.scala:403) > at scala.collection.immutable.Set$Set2.foreach(Set.scala:111) > at kafka.controller.TopicDeletionManager$DeleteTopicsThread$$anonfun$doWork$1.apply$mcV$sp(TopicDeletionManager.scala:403) > at kafka.controller.TopicDeletionManager$DeleteTopicsThread$$anonfun$doWork$1.apply(TopicDeletionManager.scala:397) > at kafka.controller.TopicDeletionManager$DeleteTopicsThread$$anonfun$doWork$1.apply(TopicDeletionManager.scala:397) > at kafka.utils.CoreUtils$.inLock(CoreUtils.scala:262) > at kafka.controller.TopicDeletionManager$DeleteTopicsThread.doWork(TopicDeletionManager.scala:397) > at kafka.utils.ShutdownableThread.run(ShutdownableThread.scala:63) > {quote} > -- This message was sent by Atlassian JIRA (v6.3.4#6332)