cassandra-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From DuyHai Doan <>
Subject Re: Using CDC Feature to Stream C* to Kafka (Design Proposal)
Date Mon, 10 Sep 2018 21:21:03 GMT
Also using Calvin means having to implement a distributed monotonic
sequence as a primitive, not trivial at all ...

On Mon, Sep 10, 2018 at 3:08 PM, Rahul Singh <>

> In response to mimicking Advanced replication in DSE. I understand the
> goal. Although DSE advanced replication does one way, those are use cases
> with limited value to me because ultimately it’s still a master slave
> design.
> I’m working on a prototype for this for two way replication between
> clusters or databases regardless of dB tech - and every variation I can get
> to comes down to some implementation of the Calvin protocol which basically
> verifies the change in either cluster , sequences it according to impact to
> underlying data, and then schedules the mutation in a predictable manner on
> both clusters / DBS.
> All that means is that I need to sequence the change before it happens so
> I can predictably ensure it’s Scheduled for write / Mutation. So I’m
> Back to square one: having a definitive queue / ledger separate from the
> individual commit log of the cluster.
> Rahul Singh
> Chief Executive Officer
> m 202.905.2818
> Anant Corporation
> 1010 Wisconsin Ave NW, Suite 250
> <,+Suite+250+%0D%0AWashington,+D.C.+20007&entry=gmail&source=g>
> Washington, D.C. 20007
> We build and manage digital business technology platforms.
> On Sep 10, 2018, 3:58 AM -0400, Dinesh Joshi <>,
> wrote:
> On Sep 9, 2018, at 6:08 AM, Jonathan Haddad <> wrote:
> There may be some use cases for it.. but I'm not sure what they are.  It
> might help if you shared the use cases where the extra complexity is
> required?  When does writing to Cassandra which then dedupes and writes to
> Kafka a preferred design then using Kafka and simply writing to Cassandra?
> From the reading of the proposal, it seems bring functionality similar to
> MySQL's binlog to Kafka connector. This is useful for many applications
> that want to be notified when certain (or any) rows change in the database
> primarily for a event driven application architecture.
> Implementing this in the database layer means there is a standard approach
> to getting a change notification stream. Downstream subscribers can then
> decide which notifications to act on.
> LinkedIn's databus is similar in functionality -
> However it is for heterogenous
> datastores.
> On Thu, Sep 6, 2018 at 1:53 PM Joy Gao <> wrote:
>> We have a* WIP design doc
>> <>* that goes
>> over this idea in details.
>> We haven't sort out all the edge cases yet, but would love to get some
>> feedback from the community on the general feasibility of this approach.
>> Any ideas/concerns/questions would be helpful to us. Thanks!
> Interesting idea. I did go over the proposal briefly. I concur with Jon
> about adding more use-cases to clarify this feature's potential use-cases.
> Dinesh

View raw message