kafka-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Daehn, Werner" <werner.da...@sap.com>
Subject Transactional Consistency beyond KIP-98 (transactional consumer)
Date Tue, 04 Jul 2017 11:29:43 GMT
While the new transactional consistency feature is definitely a move into the right direction,
I find the current design limitations quite severe. Understandable but severe.

First is the offset being the add-time, not the commit time. The consequence of that is, if
one transactional producer adds rows every once a while and does commit minutes or hours later,
all consumers with read=committed will not see any other producer's data until this one did
the commit. Will decrease the stability of the overall solution, won't it?
Would have been better to use the commit time as the offset value. Harder to implement but
more stable.

The other issue is at the consumer side. Current implementation does not provide transaction
guarantees on the consumers. Again, for logical reasons and to simplify the implementation,
but as a user you want to have both - when data is committed together you want to read the
entire transaction as one unit and complete.

Are there any plans to go beyond the current implementation?

Thanks in advance

  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message