cassandra-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Dinesh Joshi <dinesh.jo...@yahoo.com.INVALID>
Subject Re: Using CDC Feature to Stream C* to Kafka (Design Proposal)
Date Mon, 10 Sep 2018 07:58:07 GMT
> On Sep 9, 2018, at 6:08 AM, Jonathan Haddad <jon@jonhaddad.com <mailto:jon@jonhaddad.com>>
wrote:
> 
> There may be some use cases for it.. but I'm not sure what they are.  It might help if
you shared the use cases where the extra complexity is required?  When does writing to Cassandra
which then dedupes and writes to Kafka a preferred design then using Kafka and simply writing
to Cassandra?

From the reading of the proposal, it seems bring functionality similar to MySQL's binlog to
Kafka connector. This is useful for many applications that want to be notified when certain
(or any) rows change in the database primarily for a event driven application architecture.

Implementing this in the database layer means there is a standard approach to getting a change
notification stream. Downstream subscribers can then decide which notifications to act on.

LinkedIn's databus is similar in functionality - https://github.com/linkedin/databus <https://github.com/linkedin/databus>
However it is for heterogenous datastores.

> On Thu, Sep 6, 2018 at 1:53 PM Joy Gao <joyg@wepay.com.invalid <mailto:joyg@wepay.com.invalid>>
wrote:
> 
> 
> We have a WIP design doc <https://wepayinc.box.com/s/fmdtw0idajyfa23hosf7x4ustdhb0ura>
that goes over this idea in details.
> 
> We haven't sort out all the edge cases yet, but would love to get some feedback from
the community on the general feasibility of this approach. Any ideas/concerns/questions would
be helpful to us. Thanks!
> 

Interesting idea. I did go over the proposal briefly. I concur with Jon about adding more
use-cases to clarify this feature's potential use-cases.

Dinesh
Mime
View raw message