Return-Path: X-Original-To: apmail-kafka-dev-archive@www.apache.org Delivered-To: apmail-kafka-dev-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 9FF04E396 for ; Tue, 22 Jan 2013 20:18:13 +0000 (UTC) Received: (qmail 80208 invoked by uid 500); 22 Jan 2013 20:18:13 -0000 Delivered-To: apmail-kafka-dev-archive@kafka.apache.org Received: (qmail 80174 invoked by uid 500); 22 Jan 2013 20:18:12 -0000 Mailing-List: contact dev-help@kafka.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@kafka.apache.org Delivered-To: mailing list dev@kafka.apache.org Received: (qmail 80127 invoked by uid 99); 22 Jan 2013 20:18:12 -0000 Received: from arcas.apache.org (HELO arcas.apache.org) (140.211.11.28) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 22 Jan 2013 20:18:12 +0000 Date: Tue, 22 Jan 2013 20:18:12 +0000 (UTC) From: "Neha Narkhede (JIRA)" To: dev@kafka.apache.org Message-ID: In-Reply-To: References: Subject: [jira] [Updated] (KAFKA-716) SimpleConsumerPerformance does not consume all available messages MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394 [ https://issues.apache.org/jira/browse/KAFKA-716?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Neha Narkhede updated KAFKA-716: -------------------------------- Priority: Blocker (was: Major) > SimpleConsumerPerformance does not consume all available messages > ----------------------------------------------------------------- > > Key: KAFKA-716 > URL: https://issues.apache.org/jira/browse/KAFKA-716 > Project: Kafka > Issue Type: Bug > Reporter: John Fung > Priority: Blocker > > To reproduce the issue: > 1. Start 1 zookeeper > 2. Start 1 broker > 3. Send some messages > 4. Start SimpleConsumerPerformance to consume messages. The only way to consume all messages is to set the fetch-size to be greater than the log segment file size. > 5. This output shows that SimpleConsumerPerformance consumes only 6 messages: > $ bin/kafka-run-class.sh kafka.perf.SimpleConsumerPerformance --server kafka://host1:9092 --topic topic_001 --fetch-size 2048 --partition 0 > start.time, end.time, fetch.size, data.consumed.in.MB, MB.sec, data.consumed.in.nMsg, nMsg.sec > 2013-01-21 15:09:21:124, 2013-01-21 15:09:21:165, 2048, 0.0059, 0.1429, 6, 146.3415 > 6. This output shows that ConsoleConsumer consumes all 5500 messages (same test as the above) > $ bin/kafka-run-class.sh kafka.consumer.ConsoleConsumer --zookeeper host2:2181 --topic topic_001 --consumer-timeout-ms 5000 --formatter kafka.consumer.ChecksumMessageFormatter --from-beginning | grep ^checksum | wc -l > Consumed 5500 messages > 5500 -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira