Return-Path: X-Original-To: apmail-kafka-dev-archive@www.apache.org Delivered-To: apmail-kafka-dev-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id D9D1110FBD for ; Tue, 4 Nov 2014 17:40:35 +0000 (UTC) Received: (qmail 28708 invoked by uid 500); 4 Nov 2014 17:40:35 -0000 Delivered-To: apmail-kafka-dev-archive@kafka.apache.org Received: (qmail 28656 invoked by uid 500); 4 Nov 2014 17:40:35 -0000 Mailing-List: contact dev-help@kafka.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@kafka.apache.org Delivered-To: mailing list dev@kafka.apache.org Received: (qmail 28643 invoked by uid 99); 4 Nov 2014 17:40:35 -0000 Received: from arcas.apache.org (HELO arcas.apache.org) (140.211.11.28) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 04 Nov 2014 17:40:35 +0000 Date: Tue, 4 Nov 2014 17:40:35 +0000 (UTC) From: "Sriharsha Chintalapani (JIRA)" To: dev@kafka.apache.org Message-ID: In-Reply-To: References: Subject: [jira] [Commented] (KAFKA-1724) Errors after reboot in single node setup MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394 [ https://issues.apache.org/jira/browse/KAFKA-1724?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14196423#comment-14196423 ] Sriharsha Chintalapani commented on KAFKA-1724: ----------------------------------------------- [~otis] I started working on this. Will send a patch soon. > Errors after reboot in single node setup > ---------------------------------------- > > Key: KAFKA-1724 > URL: https://issues.apache.org/jira/browse/KAFKA-1724 > Project: Kafka > Issue Type: Bug > Affects Versions: 0.8.2 > Reporter: Ciprian Hacman > Assignee: Sriharsha Chintalapani > Labels: newbie > Fix For: 0.8.2 > > > In a single node setup, after reboot, Kafka logs show the following: > {code} > [2014-10-22 16:37:22,206] INFO [Controller 0]: Controller starting up (kafka.controller.KafkaController) > [2014-10-22 16:37:22,419] INFO [Controller 0]: Controller startup complete (kafka.controller.KafkaController) > [2014-10-22 16:37:22,554] INFO conflict in /brokers/ids/0 data: {"jmx_port":-1,"timestamp":"1413995842465","host":"ip-10-91-142-54.eu-west-1.compute.internal","version":1,"port":9092} stored data: {"jmx_port":-1,"timestamp":"1413994171579","host":"ip-10-91-142-54.eu-west-1.compute.internal","version":1,"port":9092} (kafka.utils.ZkUtils$) > [2014-10-22 16:37:22,736] INFO I wrote this conflicted ephemeral node [{"jmx_port":-1,"timestamp":"1413995842465","host":"ip-10-91-142-54.eu-west-1.compute.internal","version":1,"port":9092}] at /brokers/ids/0 a while back in a different session, hence I will backoff for this node to be deleted by Zookeeper and retry (kafka.utils.ZkUtils$) > [2014-10-22 16:37:25,010] ERROR Error handling event ZkEvent[Data of /controller changed sent to kafka.server.ZookeeperLeaderElector$LeaderChangeListener@a6af882] (org.I0Itec.zkclient.ZkEventThread) > java.lang.IllegalStateException: Kafka scheduler has not been started > at kafka.utils.KafkaScheduler.ensureStarted(KafkaScheduler.scala:114) > at kafka.utils.KafkaScheduler.shutdown(KafkaScheduler.scala:86) > at kafka.controller.KafkaController.onControllerResignation(KafkaController.scala:350) > at kafka.controller.KafkaController$$anonfun$2.apply$mcV$sp(KafkaController.scala:162) > at kafka.server.ZookeeperLeaderElector$LeaderChangeListener$$anonfun$handleDataDeleted$1.apply$mcZ$sp(ZookeeperLeaderElector.scala:138) > at kafka.server.ZookeeperLeaderElector$LeaderChangeListener$$anonfun$handleDataDeleted$1.apply(ZookeeperLeaderElector.scala:134) > at kafka.server.ZookeeperLeaderElector$LeaderChangeListener$$anonfun$handleDataDeleted$1.apply(ZookeeperLeaderElector.scala:134) > at kafka.utils.Utils$.inLock(Utils.scala:535) > at kafka.server.ZookeeperLeaderElector$LeaderChangeListener.handleDataDeleted(ZookeeperLeaderElector.scala:134) > at org.I0Itec.zkclient.ZkClient$6.run(ZkClient.java:549) > at org.I0Itec.zkclient.ZkEventThread.run(ZkEventThread.java:71) > [2014-10-22 16:37:28,757] INFO Registered broker 0 at path /brokers/ids/0 with address ip-10-91-142-54.eu-west-1.compute.internal:9092. (kafka.utils.ZkUtils$) > [2014-10-22 16:37:28,849] INFO [Kafka Server 0], started (kafka.server.KafkaServer) > [2014-10-22 16:38:56,718] INFO Closing socket connection to /127.0.0.1. (kafka.network.Processor) > [2014-10-22 16:38:56,850] INFO Closing socket connection to /127.0.0.1. (kafka.network.Processor) > [2014-10-22 16:38:56,985] INFO Closing socket connection to /127.0.0.1. (kafka.network.Processor) > {code} > The last log line repeats forever and is correlated with errors on the app side. > Restarting Kafka fixes the errors. > Steps to reproduce (with help from the mailing list): > # start zookeeper > # start kafka-broker > # create topic or start a producer writing to a topic > # stop zookeeper > # stop kafka-broker( kafka broker shutdown goes into WARN Session > 0x14938d9dc010001 for server null, unexpected error, closing socket connection and attempting reconnect (org.apache.zookeeper.ClientCnxn) java.net.ConnectException: Connection refused) > # kill -9 kafka-broker > # restart zookeeper and than kafka-broker leads into the the error above -- This message was sent by Atlassian JIRA (v6.3.4#6332)