incubator-cassandra-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From aaron morton <aa...@thelastpickle.com>
Subject Re: Using the commit log for external synchronization
Date Fri, 21 Sep 2012 08:56:49 GMT
The commit log is essentially internal implementation. The total size of the commit log is
restricted, and the multiple files used to represent segments are recycled. So once all the
memtables have been flushed for segment it may be overwritten. 

To archive the segments see the conf/commitlog_archiving.properties file. 

Large rows will bypass the commit log. 

A write commited to the commit log may still be considered a failure if CL nodes do not succeed.


IMHO it's a better design to multiplex the data stream at the application level.   
 
Hope that helps.

-----------------
Aaron Morton
Freelance Developer
@aaronmorton
http://www.thelastpickle.com

On 21/09/2012, at 11:51 AM, Brian O'Neill <bone@alumni.brown.edu> wrote:

> 
> Along those lines...
> 
> We sought to use triggers for external synchronization.   If you read through this issue:
> https://issues.apache.org/jira/browse/CASSANDRA-1311
> 
> You'll see the idea of leveraging a commit log for synchronization, via triggers.
> 
> We went ahead and implemented this concept in:
> https://github.com/hmsonline/cassandra-triggers
> 
> With that, via AOP, you get handed the mutation as things change.  We used it for synchronizing
SOLR.  
> 
> fwiw,
> -brian
> 
> 
> 
> On Sep 20, 2012, at 7:18 PM, Michael Kjellman wrote:
> 
>> +1. Would be a pretty cool feature
>> 
>> Right now I write once to cassandra and once to kafka.
>> 
>> On 9/20/12 4:13 PM, "Data Craftsman 木匠" <database.craftsman@gmail.com>
>> wrote:
>> 
>>> This will be a good new feature. I guess the development team don't
>>> have time on this yet.  ;)
>>> 
>>> 
>>> On Thu, Sep 20, 2012 at 1:29 PM, Ben Hood <0x6e6562@gmail.com> wrote:
>>>> Hi,
>>>> 
>>>> I'd like to incrementally synchronize data written to Cassandra into
>>>> an external store without having to maintain an index to do this, so I
>>>> was wondering whether anybody is using the commit log to establish
>>>> what updates have taken place since a given point in time?
>>>> 
>>>> Cheers,
>>>> 
>>>> Ben
>>> 
>>> 
>>> 
>>> -- 
>>> Thanks,
>>> 
>>> Charlie (@mujiang) 木匠
>>> =======
>>> Data Architect Developer 汉唐 田园牧歌DBA
>>> http://mujiang.blogspot.com
>> 
>> 
>> 'Like' us on Facebook for exclusive content and other resources on all Barracuda
Networks solutions.
>> Visit http://barracudanetworks.com/facebook
>> 
>> 
> 
> -- 
> Brian ONeill
> Lead Architect, Health Market Science (http://healthmarketscience.com)
> mobile:215.588.6024
> blog: http://weblogs.java.net/blog/boneill42/
> blog: http://brianoneill.blogspot.com/
> 


Mime
View raw message