incubator-cassandra-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jonathan Ellis <jbel...@gmail.com>
Subject Re: Commitlog Disk Full
Date Thu, 19 May 2011 17:09:01 GMT
That's basically the approach I want to take in
https://issues.apache.org/jira/browse/CASSANDRA-2427.

On Thu, May 19, 2011 at 12:00 PM, Mike Malone <mike@simplegeo.com> wrote:
> Just noticed this thread and figured I'd chime in since we've had similar
> issues with the commit log growing too large on our clusters. Tuning down
> the flush timeout wasn't really an acceptable solution for us since we
> didn't want to be constantly flushing and generating extra SSTables for no
> reason. So we wrote a small tool that we start in a static block in
> CassandraServer that periodically checks the commit log size and flushes all
> memtables if they're above some threshold.
> I've attached that code. Any feedback / improvements are more than welcome.
>
> Mike
>
> On Thu, May 12, 2011 at 11:30 AM, Sanjeev Kulkarni <sanjeev@locomatix.com>
> wrote:
>>
>> Hey guys,
>> I have a ec2 debian cluster consisting of several nodes running 0.7.5 on
>> ephimeral disks.
>> These are fresh installs and not upgrades.
>> The commitlog is set to the smaller of the disks which is around 10G in
>> size and the datadir is set to the bigger disk.
>> The config file is basically the same as the one supplied by the default
>> installation.
>> Our applications write to the cluster. After about a day of writing we
>> started noticing the commitlog disk filling up. Soon we went over the disk
>> limit and writes started failing. At this point we stopped the cluster.
>> Over the course of the day we inserted around 25G of data. Our columns
>> values are pretty small.
>> I understand that cassandra periodically cleans up the commitlog
>> directories by generating sstables in datadir. Is there any way to speed up
>> this movement from commitog to datadir?
>> Thanks!
>
>



-- 
Jonathan Ellis
Project Chair, Apache Cassandra
co-founder of DataStax, the source for professional Cassandra support
http://www.datastax.com

Mime
View raw message