gearpump-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Manu Zhang (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (GEARPUMP-40) KafkaSource poor performance
Date Wed, 20 Apr 2016 01:40:25 GMT

     [ https://issues.apache.org/jira/browse/GEARPUMP-40?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Manu Zhang updated GEARPUMP-40:
-------------------------------
    Description: 
As per [https://engineering.linkedin.com/kafka/benchmarking-apache-kafka-2-million-writes-second-three-cheap-machines],
a single thread Kafka consumer could consume at 89.7MB/s from 6x partition 3x replica topic
whose data are evenly distributed on a 3-node GbE cluster. I carried out a similar experiment
with KafkaSource and found that the throughput is only at 10MB/s.  



  was:
As per [https://engineering.linkedin.com/kafka/benchmarking-apache-kafka-2-million-writes-second-three-cheap-machines],
a single thread Kafka consumer could consume at 89.7MB/s from 6x partition 3x replica topic.
I carried out a similar experiment with KafkaSource and found that the throughput is only
at 10MB/s.  




> KafkaSource poor performance
> ----------------------------
>
>                 Key: GEARPUMP-40
>                 URL: https://issues.apache.org/jira/browse/GEARPUMP-40
>             Project: Apache Gearpump
>          Issue Type: Improvement
>          Components: kafka
>    Affects Versions: 0.8.0
>            Reporter: Manu Zhang
>            Assignee: Manu Zhang
>
> As per [https://engineering.linkedin.com/kafka/benchmarking-apache-kafka-2-million-writes-second-three-cheap-machines],
a single thread Kafka consumer could consume at 89.7MB/s from 6x partition 3x replica topic
whose data are evenly distributed on a 3-node GbE cluster. I carried out a similar experiment
with KafkaSource and found that the throughput is only at 10MB/s.  



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message