kafka-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Yang Ye (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (KAFKA-267) Enhance ProducerPerformance to generate unique random Long value for payload
Date Thu, 01 Nov 2012 08:53:12 GMT

     [ https://issues.apache.org/jira/browse/KAFKA-267?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Yang Ye updated KAFKA-267:
--------------------------

    Attachment: kafka_267_additional.diff


This is a follow up patch in order to simulate real data load. Before we generate sequential
messages padded with a series of character "x"'s, which leads to very good compression ratio
which is not typical in real case. So we want to generate random payload.

To give a comparison: sending 400 messages with batch size 200, before the data size is 48800
bytes, after the patch the size is 65600 bytes.



                
> Enhance ProducerPerformance to generate unique random Long value for payload
> ----------------------------------------------------------------------------
>
>                 Key: KAFKA-267
>                 URL: https://issues.apache.org/jira/browse/KAFKA-267
>             Project: Kafka
>          Issue Type: Improvement
>            Reporter: John Fung
>            Assignee: Yang Ye
>             Fix For: 0.8
>
>         Attachments: kafka_267_additional.diff, kafka-267-v1.patch, kafka-267-v2.patch,
kafka-267-v3.patch, kafka-267-v4.patch, kafka-267-v5.patch, kafka-267-v6.patch, kafka-267-v7.patch
>
>
> This is achieved by:
> 1. Adding a new class UniqueRandom to shuffle a range of numbers.
> 2. An optional new argument "start-index" is added to specify the starting number of
the range to be shuffled. If this argument is omitted, it is defaulted to 1. So it is backward
compatible with the argument options.
> 3. The ending number of the range is the starting number + number of messages - 1.
> Other ProducerPerformance advancement: 
> 1. producing to multiple topics
> 2. supporting multiple instances of producer performance ( and distinguishes them)
> 3. allowing waiting some time after sending a request

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message