cassandra-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Alain RODRIGUEZ <arodr...@gmail.com>
Subject Re: Increase compaction performance
Date Thu, 11 Feb 2016 16:33:13 GMT
>
> CPU load is fine, SSD disks below 30% utilization, no long GC pauses



What is your current compaction throughput ?  The current value of
'concurrent_compactors' (cassandra.yaml or through JMX) ?

nodetool getcompactionthroughput

How to speed up compaction? Increased compaction throughput and concurrent
> compactors but no change. Seems there is plenty idle resources but can't
> force C* to use it.
>

You might want to try un-throttle the compaction throughput through:

nodetool setcompactionsthroughput 0

Choose a canari node. Monitor compaction pending and disk throughput (make
sure server is ok too - CPU...)

Some other information could be useful:

What is your number of cores per machine and the compaction strategies for
the 'most compacting' tables. What are write/update patterns, any TTL or
tombstones ? Do you use a high number of vnodes ?

Also what is your repair routine and your values for gc_grace_seconds ?
When was your last repair and do you think your cluster is suffering of a
high entropy ?

You can lower the stream throughput to make sure nodes can cope with what
repairs are feeding them.

nodetool getstreamthroughput
nodetool setstreamthroughput X

C*heers,

-----------------
Alain Rodriguez
France

The Last Pickle
http://www.thelastpickle.com

2016-02-11 16:55 GMT+01:00 Michał Łowicki <mlowicki@gmail.com>:

> Hi,
>
> Using 2.1.12 across 3 DCs. Each DC has 8 nodes. Trying to run repair using
> Cassandra Reaper but nodes after couple of hours are full of pending
> compaction tasks (regular not the ones about validation)
>
> CPU load is fine, SSD disks below 30% utilization, no long GC pauses.
>
> How to speed up compaction? Increased compaction throughput and concurrent
> compactors but no change. Seems there is plenty idle resources but can't
> force C* to use it.
>
> Any clue where there might be a bottleneck?
>
>
> --
> BR,
> Michał Łowicki
>
>

Mime
View raw message