incubator-cassandra-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jonathan Ellis <jbel...@gmail.com>
Subject Re: Leveled Compaction in cassandra1.0 may not be perfect
Date Fri, 02 Dec 2011 16:48:13 GMT
I think you're confusing "temporary space used during a compaction
operation" with "total i/o done by compaction."

Leveled compaction *will* do more i/o than size-tiered, because it's
enforcing tighter guarantees on how compacted the data is.

On Fri, Dec 2, 2011 at 1:01 AM, liangfeng <liangfeng@made-in-china.com> wrote:
> Hello,everyone!
>   In this
> doc(http://www.datastax.com/dev/blog/leveled-compaction-in-apache-cassandra),I
> found a conclusion "Only enough space for 10x the sstable size needs to be
> reserved for temporary use by compaction".I don't know how can we got this
> conclusion,but I guess the author of this doc may got this conclusion by one
> compaction rule from levelDB.In one doc of
> levelDB(http://leveldb.googlecode.com/svn/trunk/doc/impl.html),I found this rule
> "We also switch to a new output file when the key range of the current output
> file has grown enough to overlap more then ten level-(L+2) files".As this
> compaction rule descripting,10x the sstable size is enough for every
> compaction.Unfortunatly,cassandra1.0 may not implement this compaction rule,so I
> think this conclusion may be arbitrary.Everyone,what do you think about it?
>   Of course,implement this compaction rule may not be hard,but this
> implementation may cause another problem.Many small sstables which overlap just
> 10 sstables with next level may be generated in compaction,especially when we
> use RandomPartitioner.This may cause many compactions when these small sstables
> have to uplevel to next level.In my practice,I write 120g data to one cassandra
> node,and cassandra node spent 24 hourse to compacte this data by Leveled
> Compaction.So,I don't think Leveled Compaction is perfect.What do you think
> about it,my friends?
>



-- 
Jonathan Ellis
Project Chair, Apache Cassandra
co-founder of DataStax, the source for professional Cassandra support
http://www.datastax.com

Mime
View raw message