incubator-cassandra-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Chris Lohfink <clohf...@blackbirdit.com>
Subject Re: high pending compactions
Date Mon, 09 Jun 2014 19:11:36 GMT
Bean: org.apache.cassandra.db.CompactionManager

also nodetool compactionstats gives you how many are in the queue + estimate of how many will
be needed.

in 1.1 you will OOM far before you hit the limit,.  In theory though, the compaction executor
is a little special cased and will actually throw an exception (normally it will block)

Chris

On Jun 9, 2014, at 7:49 AM, S C <asf11@outlook.com> wrote:

> Thank you all for valuable suggestions. Couple more questions,
> 
> How to check the compaction queue? MBean/C* system log ?
> What happens if the queue is full?
> 
> From: colinkuo.tw@gmail.com
> Date: Mon, 9 Jun 2014 18:53:41 +0800
> Subject: Re: high pending compactions
> To: user@cassandra.apache.org
> 
> As Jake suggested, you could firstly increase "compaction_throughput_mb_per_sec" and
"concurrent_compactions" to suitable values if system resource is allowed. From my understanding,
major compaction will internally acquire lock before running compaction. In your case, there
might be a major compaction blocking the pending following compaction tasks. You could check
the result of "nodetool compactionstats" and C* system log for double confirm.
> 
> If the running compaction is compacting wide row for a long time, you could try to tune
"in_memory_compaction_limit_in_mb" value.
> 
> Thanks,
> 
> 
> 
> On Sun, Jun 8, 2014 at 11:27 PM, S C <asf11@outlook.com> wrote:
> I am using Cassandra 1.1 (sorry bit old) and I am seeing high pending compaction count.
"pending tasks: 67" while active compaction tasks are not more than 5. I have a 24CPU machine.
Shouldn't I be seeing more compactions? Is this a pattern of high writes and compactions backing
up? How can I improve this? Here are my thoughts.
> 
> Increase memtable_total_space_in_mb
> Increase compaction_throughput_mb_per_sec
> Increase concurrent_compactions
> 
> Sorry if this was discussed already. Any pointers is much appreciated. 
> 
> Thanks,
> Kumar


Mime
View raw message