Mailing-List: contact dev-help@kafka.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@kafka.apache.org
Date: Sat, 2 Feb 2013 00:30:12 +0000 (UTC)
From: "Jay Kreps (JIRA)" <jira@apache.org>
To: dev@kafka.apache.org
Message-ID: <JIRA.12629289.1359131376522.233186.1359765012285@arcas>
In-Reply-To: <JIRA.12629289.1359131376522@arcas>
References: <JIRA.12629289.1359131376522@arcas>
Subject: [jira] [Commented] (KAFKA-736) Add an option to the 0.8 producer to
 mimic 0.7 producer behavior
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit


    [ https://issues.apache.org/jira/browse/KAFKA-736?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13569304#comment-13569304 ] 

Jay Kreps commented on KAFKA-736:
---------------------------------

It would be nice to have an "ordering" torture test. There are actually lots of ways ordering can get messed up (producer client code, network layer, broker, synchronization) and this would help check that invariant. This can't really be done in a unit test very well since it depends on load. This would be a command line tool that produced N messages from M threads. Each message would be of the form T-i where T is the thread number and i is a sequential counter incremented by the producer. When all N messages had been produced the tool would start a consumer and consume all the messages and keep a map of threadId => last_seen. The last_seen value should increase sequentially for each thread with no gaps.
                
> Add an option to the 0.8 producer to mimic 0.7 producer behavior
> ----------------------------------------------------------------
>
>                 Key: KAFKA-736
>                 URL: https://issues.apache.org/jira/browse/KAFKA-736
>             Project: Kafka
>          Issue Type: Improvement
>          Components: producer 
>    Affects Versions: 0.8
>            Reporter: Neha Narkhede
>            Assignee: Neha Narkhede
>            Priority: Blocker
>              Labels: p2, replication-performance
>         Attachments: kafka-736-draft.patch, kafka-736-v1.patch, kafka-736-v2.patch
>
>   Original Estimate: 24h
>  Remaining Estimate: 24h
>
> I profiled a producer throughput benchmark between a producer and a remote broker. It turns out that the background send threads spends ~97% of its time waiting to read the acknowledgement from the broker.
> I propose we change the current behavior of request.required.acks=0 to mean no acknowledgement from the broker. This will mimic the 0.7 producer behavior and will enable tuning the producer for very high throughput.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira