Return-Path: X-Original-To: archive-asf-public-internal@cust-asf2.ponee.io Delivered-To: archive-asf-public-internal@cust-asf2.ponee.io Received: from cust-asf.ponee.io (cust-asf.ponee.io [163.172.22.183]) by cust-asf2.ponee.io (Postfix) with ESMTP id 8F133200BD0 for ; Wed, 30 Nov 2016 14:37:06 +0100 (CET) Received: by cust-asf.ponee.io (Postfix) id 8DBFA160B13; Wed, 30 Nov 2016 13:37:06 +0000 (UTC) Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by cust-asf.ponee.io (Postfix) with SMTP id 09B9C160B08 for ; Wed, 30 Nov 2016 14:37:05 +0100 (CET) Received: (qmail 1376 invoked by uid 500); 30 Nov 2016 13:36:58 -0000 Mailing-List: contact dev-help@kafka.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@kafka.apache.org Delivered-To: mailing list dev@kafka.apache.org Received: (qmail 1355 invoked by uid 99); 30 Nov 2016 13:36:58 -0000 Received: from arcas.apache.org (HELO arcas) (140.211.11.28) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 30 Nov 2016 13:36:58 +0000 Received: from arcas.apache.org (localhost [127.0.0.1]) by arcas (Postfix) with ESMTP id 5F6EF2C1F54 for ; Wed, 30 Nov 2016 13:36:58 +0000 (UTC) Date: Wed, 30 Nov 2016 13:36:58 +0000 (UTC) From: "Ismael Juma (JIRA)" To: dev@kafka.apache.org Message-ID: In-Reply-To: References: Subject: [jira] [Updated] (KAFKA-4415) Reduce time to create and send MetadataUpdateRequest MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394 archived-at: Wed, 30 Nov 2016 13:37:06 -0000 [ https://issues.apache.org/jira/browse/KAFKA-4415?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ismael Juma updated KAFKA-4415: ------------------------------- Fix Version/s: (was: 0.10.1.1) 0.10.2.0 > Reduce time to create and send MetadataUpdateRequest > ---------------------------------------------------- > > Key: KAFKA-4415 > URL: https://issues.apache.org/jira/browse/KAFKA-4415 > Project: Kafka > Issue Type: Bug > Affects Versions: 0.10.1.0 > Reporter: Dong Lin > Assignee: Dong Lin > Fix For: 0.10.2.0 > > > As of current implementation, when controller receives ControlledShutdownRequest, it will 1) for every broker in the cluster, for every partition on the broker which wants to shutdown, create an instance of PartitionStateInfo and add it to ControllerBrokerRequestBatch.,updateMetadataRequestMap; and 2) for every broker, for every follower partitions on the broker which wants to shutdown, send one MetadataUpdateRequst to that broker. > In order to shutdown a broker, the controller will need to instantiate O(partitionNum * brokerNum) PartitionStateInfo and send O(partitionNum * brokerNum) partitionStateInfo. This is not efficient. The broker should only need to instantiate O(partitionNum) PartitionStateInfo and send O(brokerNum) MetadataUpdateRequest. > Micro-benchmark results show that this optimization can reduce the time of processing ControlledShutdownRequest by 30%. -- This message was sent by Atlassian JIRA (v6.3.4#6332)