incubator-cassandra-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Brian O'Neill" <b...@alumni.brown.edu>
Subject Re: Using the commit log for external synchronization
Date Sat, 22 Sep 2012 00:46:24 GMT
> IMHO it's a better design to multiplex the data stream at the application
> level.
+1, agreed.

That is where we ended up. (and Storm is proving to be a solid
framework for that)

-brian

On Fri, Sep 21, 2012 at 4:56 AM, aaron morton <aaron@thelastpickle.com> wrote:
> The commit log is essentially internal implementation. The total size of the
> commit log is restricted, and the multiple files used to represent segments
> are recycled. So once all the memtables have been flushed for segment it may
> be overwritten.
>
> To archive the segments see the conf/commitlog_archiving.properties file.
>
> Large rows will bypass the commit log.
>
> A write commited to the commit log may still be considered a failure if CL
> nodes do not succeed.
>
> IMHO it's a better design to multiplex the data stream at the application
> level.
>
> Hope that helps.
>
> -----------------
> Aaron Morton
> Freelance Developer
> @aaronmorton
> http://www.thelastpickle.com
>
> On 21/09/2012, at 11:51 AM, Brian O'Neill <bone@alumni.brown.edu> wrote:
>
>
> Along those lines...
>
> We sought to use triggers for external synchronization.   If you read
> through this issue:
> https://issues.apache.org/jira/browse/CASSANDRA-1311
>
> You'll see the idea of leveraging a commit log for synchronization, via
> triggers.
>
> We went ahead and implemented this concept in:
> https://github.com/hmsonline/cassandra-triggers
>
> With that, via AOP, you get handed the mutation as things change.  We used
> it for synchronizing SOLR.
>
> fwiw,
> -brian
>
>
>
> On Sep 20, 2012, at 7:18 PM, Michael Kjellman wrote:
>
> +1. Would be a pretty cool feature
>
> Right now I write once to cassandra and once to kafka.
>
> On 9/20/12 4:13 PM, "Data Craftsman 木匠" <database.craftsman@gmail.com>
> wrote:
>
> This will be a good new feature. I guess the development team don't
>
> have time on this yet.  ;)
>
>
>
> On Thu, Sep 20, 2012 at 1:29 PM, Ben Hood <0x6e6562@gmail.com> wrote:
>
> Hi,
>
>
> I'd like to incrementally synchronize data written to Cassandra into
>
> an external store without having to maintain an index to do this, so I
>
> was wondering whether anybody is using the commit log to establish
>
> what updates have taken place since a given point in time?
>
>
> Cheers,
>
>
> Ben
>
>
>
>
> --
>
> Thanks,
>
>
> Charlie (@mujiang) 木匠
>
> =======
>
> Data Architect Developer 汉唐 田园牧歌DBA
>
> http://mujiang.blogspot.com
>
>
>
> 'Like' us on Facebook for exclusive content and other resources on all
> Barracuda Networks solutions.
> Visit http://barracudanetworks.com/facebook
>
>
>
> --
> Brian ONeill
> Lead Architect, Health Market Science (http://healthmarketscience.com)
> mobile:215.588.6024
> blog: http://weblogs.java.net/blog/boneill42/
> blog: http://brianoneill.blogspot.com/
>
>



-- 
Brian ONeill
Lead Architect, Health Market Science (http://healthmarketscience.com)
Apache Cassandra MVP
mobile:215.588.6024
blog: http://brianoneill.blogspot.com/
twitter: @boneill42

Mime
View raw message