kafka-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From 郭旭 <guoxu1...@gmail.com>
Subject Message sent ordering guarantees
Date Thu, 01 Sep 2016 03:06:16 GMT
Hi Kafka Experts,

(Sorry to send this question to DEV group, but it seems that I can not find
related document in user manual.)

For official document ,I can find message sent guarantee as below.
For *sync producer*, I think it is true but sync sent are very slow.(about
408 message per second if ack = all, 1000 message per second if ack = 1).

batch and async sent could satisfy our throughput requirement, but I'm not
sure if message sent ordering are guaranteed in *async *style.

For some critical application, for example( replicate mysql binlog to kafka
distributed committed log), binlog ordering are important(partitioned by
database/table/PK). throughput also important.

If I use async producer, partition the binlog by table and send them in
batch. Is it safe for binlog ordering for a single table?

Will async producer guarantee the send ordering?


Regards
Shawn

Guarantees <http://kafka.apache.org/documentation.html#intro_guarantees>At
a high-level Kafka gives the following guarantees:

   - Messages sent by a producer to a particular topic partition will be
   appended in the order they are sent. That is, if a message M1 is sent by
   the same producer as a message M2, and M1 is sent first, then M1 will have
   a lower offset than M2 and appear earlier in the log.

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message