Return-Path: X-Original-To: archive-asf-public-internal@cust-asf2.ponee.io Delivered-To: archive-asf-public-internal@cust-asf2.ponee.io Received: from cust-asf.ponee.io (cust-asf.ponee.io [163.172.22.183]) by cust-asf2.ponee.io (Postfix) with ESMTP id D6819200BEA for ; Tue, 27 Dec 2016 15:31:22 +0100 (CET) Received: by cust-asf.ponee.io (Postfix) id D54A3160B31; Tue, 27 Dec 2016 14:31:22 +0000 (UTC) Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by cust-asf.ponee.io (Postfix) with SMTP id 29431160B23 for ; Tue, 27 Dec 2016 15:31:22 +0100 (CET) Received: (qmail 64425 invoked by uid 500); 27 Dec 2016 14:31:19 -0000 Mailing-List: contact dev-help@kafka.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@kafka.apache.org Delivered-To: mailing list dev@kafka.apache.org Received: (qmail 64401 invoked by uid 99); 27 Dec 2016 14:31:19 -0000 Received: from pnap-us-west-generic-nat.apache.org (HELO spamd3-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 27 Dec 2016 14:31:19 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd3-us-west.apache.org (ASF Mail Server at spamd3-us-west.apache.org) with ESMTP id 114F3180BC1; Tue, 27 Dec 2016 14:31:19 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd3-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: 1.98 X-Spam-Level: * X-Spam-Status: No, score=1.98 tagged_above=-999 required=6.31 tests=[HEADER_FROM_DIFFERENT_DOMAINS=0.001, HTML_MESSAGE=2, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H3=-0.01, RCVD_IN_MSPIKE_WL=-0.01, SPF_PASS=-0.001] autolearn=disabled Received: from mx1-lw-us.apache.org ([10.40.0.8]) by localhost (spamd3-us-west.apache.org [10.40.0.10]) (amavisd-new, port 10024) with ESMTP id 9Vu6KmFeaABB; Tue, 27 Dec 2016 14:31:13 +0000 (UTC) Received: from mail-pg0-f51.google.com (mail-pg0-f51.google.com [74.125.83.51]) by mx1-lw-us.apache.org (ASF Mail Server at mx1-lw-us.apache.org) with ESMTPS id 1011C5F24B; Tue, 27 Dec 2016 14:31:13 +0000 (UTC) Received: by mail-pg0-f51.google.com with SMTP id f188so145663954pgc.3; Tue, 27 Dec 2016 06:31:13 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:message-id:subject:mime-version; bh=OpDzWyqriOb5OyEjQToGfQkKD60K7E3FS+cMjT/nnY0=; b=nqyQXLTF6BSzOS3ClY0Y99rZr7OFBo+0gXn8Uqwpxj5qxtAmKghWm3Xie7ZDb/dgas cv63Eecie7PdGNLD7yJ7tZKH11iA2nDTN1C4h9eb6qh0RRDGqMZupj/EVpzz9ba/0BrZ R15C/dNcT3yS2505LC6L2S+hniarHQV6aEir9ceQxeCQKVd00xaS3dljxcPtFpQ9q30F zDDJaXJ+Lt9+8mB6oHbpKWjP8ch+oSaaSW0a6JoXWnu4j0hfG03s9EOyjDhe0dDbACVE 4xEBQL1MIfUQYpdYRFnJmIB+bymS2qj2ED3AFZLYRrk7aVanPuDZE2ZxG7Hgzugy5SNv g0/g== X-Gm-Message-State: AIkVDXJTD1SASXc9R9VWj0Zwn1gEcmYgb6AFohq8i9rv4YFmYP2HyPc7QBEUcxBmGHYNwA== X-Received: by 10.99.228.5 with SMTP id a5mr59502977pgi.1.1482849063146; Tue, 27 Dec 2016 06:31:03 -0800 (PST) Received: from mail.outlook.com (ec2-52-32-248-203.us-west-2.compute.amazonaws.com. [52.32.248.203]) by smtp.gmail.com with ESMTPSA id o29sm90930560pgn.28.2016.12.27.06.31.01 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 27 Dec 2016 06:31:02 -0800 (PST) Date: Tue, 27 Dec 2016 14:31:01 +0000 (UTC) From: Dongjin Lee To: "dev@kafka.apache.org" , "users@kafka.apache.org" Message-ID: Subject: Experiencing trouble with KafkaCSVMetricsReporter MIME-Version: 1.0 Content-Type: multipart/alternative; boundary="----=_Part_7330_1848307511.1482849061290" X-Mailer: Outlook for iOS and Android archived-at: Tue, 27 Dec 2016 14:31:23 -0000 ------=_Part_7330_1848307511.1482849061290 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit In short: the resulting csv files from the brokers are filled with 0 only, although the broker cluster is running correctly. Hello. I am trying some benchmarks with KAFKA-4514[^1]. However, My KafkaCSVMetricsReporter is not working properly. I would like to ask if someone on this mailing list has experienced similar cases. Let me explain. I configured a Kafka cluster with 3 zookeeper instances and 3 Kafka Broker instances. After confirming all is working correctly, I added following properties in server.properties file and restarted the brokers. > kafka.metrics.polling.interval.secs=5> kafka.metrics.reporters=kafka.metrics.KafkaCSVMetricsReporter> kafka.csv.metrics.reporter.enabled=true By re-running the brokers and the producer, I acquired some csv files from kafka_metrics directory. But there are two weird things: 1. Kafka brokers repeatedly try to create csv files and fail with IO Exception. Why they try to create files, yet the resulting csv files already exist?2. All cells of the resulting csv files are filled with 0, except their headers. Since the producer generated so many messages, they (e.g., BytesPerSec) cannot be 0. Any pieces of advices or comments are welcome. Thanks in advance. Regards,Dongjin [^1]: https://issues.apache.org/jira/browse/KAFKA-4514 ------=_Part_7330_1848307511.1482849061290--