cassandra-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jay Zhuang <jay.zhu...@yahoo.com.INVALID>
Subject Re: commitlog_total_space_in_mb tuning
Date Wed, 05 Jul 2017 23:35:54 GMT
Thanks Jeff for the quick response.

We're running with 3.0.13 which doesn't have commitlog_segment_recycling
option. So it should be disabled.

I think the CL flush is because commitlog full. The commitlog size is
closing to 8G:
#mbean =
org.apache.cassandra.metrics:name=TotalCommitLogSize,type=CommitLog:
Value = 8489271296;

We're running with 128G memory and 30G heap size. Maybe it's good idea
to increase the commitlog_total_space. On the other hand, even with  8G
commitlog_total_space, replaying CL after restart takes more than 5 minutes.

In our case, the actual problem is it's causing lots of read repair
timeouts as the repair mutations are dropped. Which causes Cassandra JVM
hang or sometimes crash.

/Jay

On 7/5/17 2:45 PM, Jeff Jirsa wrote:
> 
> 
> On 2017-07-05 14:32 (-0700), Jay Zhuang <jay.zhuang@yahoo.com.INVALID> wrote: 
>> Hi,
>>
>> commitlog_total_space_in_mb is increased from 1G to 8G in
>> CASSANDRA-7031. Sometimes we saw the number of dropped mutations spikes.
>> Not sure if it's a sign that we should increase the
>> commitlog_total_space_in_mb?
> 
> 8G seems pretty typical. The real litmus test is in Benedict's comment:
> 
>> We can find that we force memtable flushes as a result of log utilisation instead
of memtable occupancy quite often
> 
> Are your memtable flushes because the memtable is full, or because the commitlog is full?
> 
> Also, what version are you running, and do you have segment recycling enabled? (hopefully
not: https://issues.apache.org/jira/browse/CASSANDRA-8771 )
> 
> 
> 
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: user-unsubscribe@cassandra.apache.org
> For additional commands, e-mail: user-help@cassandra.apache.org
> 

---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@cassandra.apache.org
For additional commands, e-mail: user-help@cassandra.apache.org


Mime
View raw message