cassandra-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Léo FERLIN SUTTON <lfer...@mailjet.com.INVALID>
Subject Re: Speed up compaction
Date Thu, 13 Jun 2019 12:07:04 GMT
On Thu, Jun 13, 2019 at 12:09 PM Oleksandr Shulgin <
oleksandr.shulgin@zalando.de> wrote:

> On Thu, Jun 13, 2019 at 11:28 AM Léo FERLIN SUTTON
> <lferlin@mailjet.com.invalid> wrote:
>
>>
>> ## Cassandra configuration :
>> 4 concurrent_compactors
>> Current compaction throughput: 150 MB/s
>> Concurrent reads/write are both set to 128.
>>
>> I have also temporarily stopped every repair operations.
>>
>> Any ideas about how I can speed this up ?
>>
>
> Hi,
>
> What is the compaction strategy used by this column family?
>
> Do you observe this behavior on one of the nodes only?  Have you tried to
> cancel this compaction and see if a new one is started and makes better
> progress?  Can you try to restart the affected node?
>
> Regards,
> --
> Alex
>
> I can't believe I forgot that information.

 Overall we are talking about a 1.08TB table, using LCS.

SSTable count: 1047
> SSTables in each level: [15/4, 10, 103/100, 918, 0, 0, 0, 0, 0]

SSTable Compression Ratio: 0.5192269874287099

Number of partitions (estimate): 7282253587


We have recently (about a month ago) deleted about 25% of the data in that
table.

Letting Cassandra reclaim the disk space on it's own (via regular
compactions) was too slow for us, so we wanted to force a compaction on the
table to reclaim the disk space faster.

The speed of the compaction doesn't seem out of the ordinary for the
cluster, only before we haven't had such a big compaction and the speed
alarmed us.
We never have a big compaction backlog, most of the time less than 5
pending tasks (per node)

Finally but we are running Cassandra 3.0.18 and plan to upgrade to 3.11 as
soon as our compactions are over.

Regards,

Leo

Mime
View raw message