cassandra-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Clint Martin <clintlmar...@coolfiretechnologies.com>
Subject Re: Publishing from cassandra
Date Sun, 24 Apr 2016 18:17:56 GMT
I would probably use a messaging layer to perform this operation. Kafka
works very well, but depending on your throughput requirements almost
anything should work.

The idea is to publish your mutation requests to the messaging layer and
allow multiple consumers to process those mutation requests into each of
your db layers (Cassandra and kdb)

I'm certain you can get something to work in the way you originally
intended, possibly using Cassandra triggers.. Or maybe using batch
processing through spark. However just about anything you implement will
incur performance overhead for Cassandra and may impede your ability to
scale.

I tend to believe in using the right tool for the job, and I don't believe
that replication from Cassandra is the right way to go here.

Separating your replication mechanism from your persistence mechanism
allows you to decouple your system and manage or replace reach part of it
independently.

Clint

On Apr 24, 2016 10:51 AM, "Alexander Orr" <mental2k@gmail.com> wrote:

> Hi,
>
> I'm wondering if someone could help me, I'd like to use cassandra to store
> data and publish this on dowstream to another database (kdb if anyone is
> interested). Essentially I'd like to be able to run a function or operation
> on cassandra from an upstream process that would insert to table and
> publish the data on downstream.
>
> I can't see anything in the docs, but I'm relatively new to cassandra.
> Assuming there's not something simple already in place what would be the
> best way to impliment this kind of mechanism? I have some java that will
> allow me to talk to the db I want to, but I'm not sure of the  best way to
> integrate this with cassandra.
>
> UDFs seem to have ponential, but I don't think it's possible to use
> external libraries/classes within UDFs. All I can think of at the minute is
> either having a process which controls cassandra, publishes to it and also
> the downstream system directly or cloning the git repo and seeing if I can
> hack in some extra functionality.
>
> Any suggestions welcome.
>
> Thanks
>
> Alex
>

Mime
View raw message