incubator-cassandra-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Terje Marthinussen <tmarthinus...@gmail.com>
Subject Re: compaction strategy
Date Sat, 07 May 2011 17:20:48 GMT
This is an all ssd system. I have no problems with read/write performance
due to I/O.
I do have a potential with the crazy explosion you can get in terms of disk
use if compaction cannot keep up.

As things falls behind and you get many generations of data, yes, read
performance gets a problem due to the number of sstables.

As things start falling behind, you have a bunch of minor compactions trying
to merge 20MB (sstables cassandra generally dumps with current config when
under pressure) into 40 MB into 80MB into....

Anyone wants to do the math on how many times you are rewriting the data
going this route?

There is just no way this can keep up. It will just fall more and more
behind.
Only way to recover as I can see would be to trigger a full compaction?

It does not really make sense to me to go through all these minor merges
when a full compaction will do a much faster and better job.

Terje

On Sat, May 7, 2011 at 9:54 PM, Jonathan Ellis <jbellis@gmail.com> wrote:

> On Sat, May 7, 2011 at 2:01 AM, Terje Marthinussen
> <tmarthinussen@gmail.com> wrote:
> > 1. Would it make sense to make full compactions occur a bit more
> aggressive.
>
> I'd rather reduce the performance impact of being behind, than do more
> full compactions: https://issues.apache.org/jira/browse/CASSANDRA-2498
>
> > 2. I
> > would think the code should be smart enough to either trigger a full
> > compaction and scrap the current queue, or at least merge some of those
> > pending tasks into larger ones
>
> Not crazy but a queue-rewriter would be nontrivial. For now I'm okay
> with saying "add capacity until compaction can mostly keep up." (Most
> people's problem is making compaction LESS aggressive, hence
> https://issues.apache.org/jira/browse/CASSANDRA-2156.)
>
> --
> Jonathan Ellis
> Project Chair, Apache Cassandra
> co-founder of DataStax, the source for professional Cassandra support
> http://www.datastax.com
>

Mime
View raw message