Return-Path: X-Original-To: apmail-kafka-commits-archive@www.apache.org Delivered-To: apmail-kafka-commits-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id A608810EDD for ; Mon, 8 Jul 2013 23:14:27 +0000 (UTC) Received: (qmail 86728 invoked by uid 500); 8 Jul 2013 23:14:27 -0000 Delivered-To: apmail-kafka-commits-archive@kafka.apache.org Received: (qmail 86688 invoked by uid 500); 8 Jul 2013 23:14:27 -0000 Mailing-List: contact commits-help@kafka.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@kafka.apache.org Delivered-To: mailing list commits@kafka.apache.org Received: (qmail 86556 invoked by uid 99); 8 Jul 2013 23:14:27 -0000 Received: from tyr.zones.apache.org (HELO tyr.zones.apache.org) (140.211.11.114) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 08 Jul 2013 23:14:27 +0000 Received: by tyr.zones.apache.org (Postfix, from userid 65534) id EE853888A77; Mon, 8 Jul 2013 23:14:26 +0000 (UTC) Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit From: junrao@apache.org To: commits@kafka.apache.org Date: Mon, 08 Jul 2013 23:14:35 -0000 Message-Id: <9255c5992ff440909413620652e45036@git.apache.org> In-Reply-To: <8ae0e5185c9e4c1793ea47489314bce7@git.apache.org> References: <8ae0e5185c9e4c1793ea47489314bce7@git.apache.org> X-Mailer: ASF-Git Admin Mailer Subject: [10/43] git commit: KAFKA-831 Controller does not send the complete list of partitions to a newly started broker; reviewed by Jun Rao KAFKA-831 Controller does not send the complete list of partitions to a newly started broker; reviewed by Jun Rao Project: http://git-wip-us.apache.org/repos/asf/kafka/repo Commit: http://git-wip-us.apache.org/repos/asf/kafka/commit/1d5e95f6 Tree: http://git-wip-us.apache.org/repos/asf/kafka/tree/1d5e95f6 Diff: http://git-wip-us.apache.org/repos/asf/kafka/diff/1d5e95f6 Branch: refs/heads/trunk Commit: 1d5e95f6c4067884a374deddd88ec3f471664658 Parents: 66b1038 Author: Neha Narkhede Authored: Thu Mar 28 16:29:37 2013 -0700 Committer: Neha Narkhede Committed: Thu Mar 28 16:29:37 2013 -0700 ---------------------------------------------------------------------- core/src/main/scala/kafka/controller/KafkaController.scala | 7 +++++-- 1 file changed, 5 insertions(+), 2 deletions(-) ---------------------------------------------------------------------- http://git-wip-us.apache.org/repos/asf/kafka/blob/1d5e95f6/core/src/main/scala/kafka/controller/KafkaController.scala ---------------------------------------------------------------------- diff --git a/core/src/main/scala/kafka/controller/KafkaController.scala b/core/src/main/scala/kafka/controller/KafkaController.scala index 9d32901..47d4d7b 100644 --- a/core/src/main/scala/kafka/controller/KafkaController.scala +++ b/core/src/main/scala/kafka/controller/KafkaController.scala @@ -273,9 +273,12 @@ class KafkaController(val config : KafkaConfig, zkClient: ZkClient) extends Logg info("New broker startup callback for %s".format(newBrokers.mkString(","))) val newBrokersSet = newBrokers.toSet - // update partition state machine - partitionStateMachine.triggerOnlinePartitionStateChange() + // the very first thing to do when a new broker comes up is send it the entire list of partitions that it is + // supposed to host. Based on that the broker starts the high watermark threads for the input list of partitions replicaStateMachine.handleStateChanges(getAllReplicasOnBroker(zkClient, controllerContext.allTopics.toSeq, newBrokers), OnlineReplica) + // when a new broker comes up, the controller needs to trigger leader election for all new and offline partitions + // to see if these brokers can become leaders for some/all of those + partitionStateMachine.triggerOnlinePartitionStateChange() // check if reassignment of some partitions need to be restarted val partitionsWithReplicasOnNewBrokers = controllerContext.partitionsBeingReassigned.filter{