kafka-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Tauzell, Dave" <Dave.Tauz...@surescripts.com>
Subject RE: Handling 2 to 3 Million Events before Kafka
Date Wed, 21 Jun 2017 13:23:36 GMT
What are your configurations?

- production
- brokers
- consumers

Is the problem that web servers cannot send to Kafka fast enough or your consumers cannot
process messages off of kafka fast enough?
What is the average size of these messages?


-----Original Message-----
From: SenthilKumar K [mailto:senthilec566@gmail.com]
Sent: Wednesday, June 21, 2017 7:58 AM
To: users@kafka.apache.org
Cc: senthilec566@apache.org; Senthil kumar; dev@kafka.apache.org
Subject: Handling 2 to 3 Million Events before Kafka

Hi Team ,   Sorry if this question is irrelevant to Kafka Group ...

I have been trying to solve problem of handling 5 GB/sec ingestion. Kafka is really good candidate
for us to handle this ingestion rate ..

100K machines ----> { Http Server (Jetty/Netty) } --> Kafka Cluster..

I see the problem in Http Server where it can't handle beyond 50K events per instance .. 
I'm thinking some other solution would be right choice before Kafka ..

Anyone worked on similar use case and similar load ? Suggestions/Thoughts ?

This e-mail and any files transmitted with it are confidential, may contain sensitive information,
and are intended solely for the use of the individual or entity to whom they are addressed.
If you have received this e-mail in error, please notify the sender by reply e-mail immediately
and destroy all copies of the e-mail and any attachments.
View raw message