From dev-return-100737-archive-asf-public=cust-asf.ponee.io@kafka.apache.org Thu Jan 3 13:36:55 2019 Return-Path: X-Original-To: archive-asf-public@cust-asf.ponee.io Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by mx-eu-01.ponee.io (Postfix) with SMTP id 56F6A180608 for ; Thu, 3 Jan 2019 13:36:54 +0100 (CET) Received: (qmail 57577 invoked by uid 500); 3 Jan 2019 12:36:53 -0000 Mailing-List: contact dev-help@kafka.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@kafka.apache.org Delivered-To: mailing list dev@kafka.apache.org Received: (qmail 57565 invoked by uid 99); 3 Jan 2019 12:36:52 -0000 Received: from pnap-us-west-generic-nat.apache.org (HELO spamd3-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 03 Jan 2019 12:36:52 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd3-us-west.apache.org (ASF Mail Server at spamd3-us-west.apache.org) with ESMTP id 071B3180A69 for ; Thu, 3 Jan 2019 12:36:52 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd3-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: 1.797 X-Spam-Level: * X-Spam-Status: No, score=1.797 tagged_above=-999 required=6.31 tests=[DKIMWL_WL_MED=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, HTML_MESSAGE=2, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-0.001, SPF_PASS=-0.001] autolearn=disabled Authentication-Results: spamd3-us-west.apache.org (amavisd-new); dkim=pass (2048-bit key) header.d=gmail.com Received: from mx1-lw-us.apache.org ([10.40.0.8]) by localhost (spamd3-us-west.apache.org [10.40.0.10]) (amavisd-new, port 10024) with ESMTP id JSIhF7lXc5BU for ; Thu, 3 Jan 2019 12:36:49 +0000 (UTC) Received: from mail-it1-f181.google.com (mail-it1-f181.google.com [209.85.166.181]) by mx1-lw-us.apache.org (ASF Mail Server at mx1-lw-us.apache.org) with ESMTPS id AA8386100E for ; Thu, 3 Jan 2019 12:36:49 +0000 (UTC) Received: by mail-it1-f181.google.com with SMTP id m8so41118870itk.0 for ; Thu, 03 Jan 2019 04:36:49 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to; bh=tYjqyOVKZJaR/gwqAtqtVYBHfFSUpGWPRETuuc7DTs0=; b=Kxf70eTpt+oXCXzwZ638q4KbMD3nOoruU5swa2erYQizyKIx1fgsfcVxYkqE21KEMV 9qfucAthjFcSZMHg06lzRBoQ9AU96ZVAnJGf9qQxcY0Oc6QF811Endlm/mQ76nNv9Bz0 FuFJnRsbHkMWKDNcouPxjPDgDksS0f/db1q8Xm5ABs252IqrX8hZ/8GdUTHA64/AYPEa BLsx4qBKDykrljqIFtGV5thqk4rp1UWNeudP04EUC4MedqNkNZLvnMIO8PBw95jwa77T rQlK8Jv3VaeIH5VQ1j1O5YKL5LhnN6pK+wNRkrSBfEmZ9Pa9GBFR2tWnNhcmhzwC0jtc QuXw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to; bh=tYjqyOVKZJaR/gwqAtqtVYBHfFSUpGWPRETuuc7DTs0=; b=gpEC4HsbvywInaR0D/lYZ61XDAPqbfUkkLkuHEnyByOlQRS0lzX6r+hm9PEUSTsf5v 6JC2aJ8Uba7eSY5eHh76Nnq7c54C0+84OxKbc/p+zJld8pGIbGcflre6o9NHasGvH4C3 V4rSoHuS3yDFu+qQOFTypU5LlnEg+RP7u6TP276jUMnUe6ehp0yELkF76Ng3al/Cljm5 scc5rm2nGoWS0sHHuQMI2IGYYNGBlEtLurmsWILY5Z/8H8HjK1ZdvfJq8QmVLHK4WMxt v/MYAgi//arhKX/F0bndJRrbtyIBe9TUhAWmOHNY2txUFYI6XJjxkLG4Zad7gd4L6tx6 xUNw== X-Gm-Message-State: AJcUukcwRO83ByAlFV+px4VyVuIEq70OwF4Nvgz9R1yRwetx/TmgYb1I LnDQJfivfSCEjVUXhaiiNPQvw6tCCRHk6vGWUgu72i0A X-Google-Smtp-Source: AFSGD/WAElLnrllEZ37wszOd0EzxBUMpR2XnKeBQNrFGSocFYls03PWm9vTaFf+f2YvGi8TdQ/F7zaFBb5t1yiA2KT0= X-Received: by 2002:a24:d60a:: with SMTP id o10mr29302675itg.162.1546519008574; Thu, 03 Jan 2019 04:36:48 -0800 (PST) MIME-Version: 1.0 References: In-Reply-To: From: Rajini Sivaram Date: Thu, 3 Jan 2019 12:36:37 +0000 Message-ID: Subject: =?UTF-8?Q?Re=3A_Use_KafkaAdminClient_to_get_broker_dynamic_confi?= =?UTF-8?Q?g_=E2=80=9Cleader=2Ereplication=2Ethrottled=2Erate=E2=80=9D_and_=E2=80=9Cfollower=2Ere?= =?UTF-8?Q?plication=2Ethrottled=2Erate=E2=80=9D_=EF=BC=8Cbut_returned_null?= To: dev Content-Type: multipart/alternative; boundary="00000000000025085d057e8d0425" --00000000000025085d057e8d0425 Content-Type: text/plain; charset="UTF-8" KafkaAdminClent#describeConfigs currently only returns values of non-sensitive configs that are defined in `KafkaConfig`. Since ` leader.replication.throttled.rate` is not defined in `KafkaConfig`, it is handled as a custom config and hence the value is not returned. We could however change this behaviour for the three configs defined in kafka.server.DynamicConfig since we know that these are not sensitive values. Can you open a JIRA? Thanks, Rajini On Thu, Jan 3, 2019 at 2:34 AM tinawenqiao <315524513@qq.com> wrote: > Hi, everyone: I found KafkaAdminClient.describeConfigs can't return the > dynamic broker config value. > First I use kafka-reassign-partitions.sh to reassign partitions, and > at the same time I add throttle as follows. And then use describeConfig to > get the config value, but I can't get the result as I expected. > Is it a bug or the design is just right like this? > > > Step 1: Reassign with throttle parameter > bin/kafka-reassign-partitions.sh --zookeeper localhost:2183 > --reassignment-json-file reassign.json --execute --throttle 3000 > --replica-alter-log-dirs-throttle 2000 --bootstrap-server localhost:19092 > Current partition replica assignment > > {"version":1,"partitions":[{"topic":"test-topic3","partition":0,"replicas":[115],"log_dirs":["any"]}]} > Save this to use as the --reassignment-json-file option during rollback > Warning: You must run Verify periodically, until the reassignment > completes, to ensure the throttle is removed. You can also alter the > throttle by rerunning the Execute command passing a new value. > The inter-broker throttle limit was set to 3000 B/s > The replica-alter-dir throttle limit was set to 2000 B/s > Successfully started reassignment of partitions. > > > > Step 2: I can get broker dynamic config on zk. > [zk: localhost:2183(CONNECTED) 47] get /config/brokers/111 > > {"version":1,"config":{"replica.alter.log.dirs.io.max.bytes.per.second":"2000","leader.replication.throttled.rate":"3000","follower.replication.throttled.rate":"3000"}} > cZxid = 0x1a99 > ctime = Fri Dec 21 13:59:20 CST 2018 > mZxid = 0x2ebb > mtime = Thu Jan 03 10:06:56 CST 2019 > pZxid = 0x1a99 > cversion = 0 > dataVersion = 19 > aclVersion = 0 > ephemeralOwner = 0x0 > dataLength = 168 > numChildren = 0 > > > > > Step3:Use org.apache.kafka.clients.admin.KafkaAdminClient describe broker > 111's config, the results are all null > leader.replication.throttled.rate = null, isReadOnly:true, > source:DYNAMIC_BROKER_CONFIG > > follower.replication.throttled.rate = null, isReadOnly:true, > source:DYNAMIC_BROKER_CONFIG > > replica.alter.log.dirs.io.max.bytes.per.second = null, isReadOnly:true, > source:DYNAMIC_BROKER_CONFIG --00000000000025085d057e8d0425--