kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Apoorva Gaurav <apoorvagau...@gmail.com>
Subject Transferring events across data centers
Date Sat, 02 Feb 2013 02:47:37 GMT
Hello All,

We are working on Kafka based event collection system. This needs to gather
events from across data centers. Lets say all the events will be produced
in DC1 while kafka brokers and consumers are lying in DC2. Round trip
between DC1 and DC2 can be around ~80 ms. Number of events should be around
~50 million a day, peak being ~5K events a day, data volume ~100GB a day,
peak being ~10MB a day. What is the best way to do it.

--- Is keeping the producer is DC1 sending events to DC2 a good idea.
--- Should my ZK quorum lie only in DC2 or should it spawn across both DC1
and DC2.
--- Will this problem be solved easily in version .8.0 through broker
replication by keeping brokers in both DC1 and DC2.

Thanks & Regards,

  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message