cassandra-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jeff Jirsa <jeff.ji...@crowdstrike.com>
Subject Re: Question about compaction strategy changes
Date Mon, 24 Oct 2016 13:38:49 GMT
 

If you drop window size, you may force some window-major compactions (if you go from 1 week
windows to 1 day windows, you’ll have 6 days worth of files start compacting into 1-day
sstables).

If you increase window size, you’ll likely have adjacent windows join (if you go from 1
day windows to 2 day windows, nearly every sstable will be joined with the one in the day
adjacent to it).

 

Short of altering compaction strategies, it seems unlikely that you’d see huge jumps where
you’d run out of space. How many tables/CFs have TWCS enabled? How much space are you using,
and how much is free?  Do you have hundreds with the same TWCS parameters? 

 

If you’re running very close to your capacity, you may want to consider dropping concurrent
compactors down so fewer compaction tasks run at the same time. That will translate proportionally
to the amount of extra disk you have consumed by compaction in a TWCS setting. 

 

 

 

From: Seth Edwards <seth@pubnub.com>
Reply-To: "user@cassandra.apache.org" <user@cassandra.apache.org>
Date: Sunday, October 23, 2016 at 7:03 PM
To: user <user@cassandra.apache.org>
Subject: Re: Question about compaction strategy changes

 

More compactions meaning "rows to be compacted" or actual number of pending compactions? I
assumed when I run nodetool compactionstats the number of pending tasks would line up with
number of sstables that will be compacted. Most of the time this is idle, then we hit spots
when it could jump into the thousands and we and up being short of a few hundred GB of disk
space. 

 

On Sun, Oct 23, 2016 at 5:49 PM, kurt Greaves <kurt@instaclustr.com> wrote:

 

On 22 October 2016 at 03:37, Seth Edwards <seth@pubnub.com> wrote:

We're using TWCS and we notice that if we make changes to the options to the window unit or
size, it seems to implicitly start recompacting all sstables.

 

If you increase the window unit or size you potentially increase the number of SSTable candidates
for compaction inside each window, which is why you would see more compactions. If you decrease
the window you shouldn't see any new compactions kicked off, however be aware that you will
have SSTables covering multiple windows, so until a full cycle of your TTL passes your read
queries won't benefit from the smaller window size.


Kurt Greaves 

kurt@instaclustr.com

www.instaclustr.com

 

CONFIDENTIALITY NOTE: This e-mail and any attachments are confidential and may be legally
privileged. If you are not the intended recipient, do not disclose, copy, distribute, or use
this email or any attachments. If you have received this in error please let the sender know
and then delete the email and all attachments.

Mime
View raw message