incubator-cassandra-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Aaron Turner <synfina...@gmail.com>
Subject Re: any ways to have compaction use less disk space?
Date Sun, 23 Sep 2012 19:24:29 GMT
On Sun, Sep 23, 2012 at 8:18 PM, Віталій Тимчишин <tivv00@gmail.com> wrote:
> If you think about space, use Leveled compaction! This won't only allow you
> to fill more space, but also will shrink you data much faster in case of
> updates. Size compaction can give you 3x-4x more space used than there are
> live data. Consider the following (our simplified) scenario:
> 1) The data is updated weekly
> 2) Each week a large SSTable is written (say, 300GB) after full update
> processing.
> 3) In 3 weeks you will have 1.2TB of data in 3 large SSTables.
> 4) Only after 4th week they all will be compacted into one 300GB SSTable.
>
> Leveled compaction've tamed space for us. Note that you should set
> sstable_size_in_mb to reasonably high value (it is 512 for us with ~700GB
> per node) to prevent creating a lot of small files.

512MB per sstable?  Wow, that's freaking huge.  From my conversations
with various developers 5-10MB seems far more reasonable.   I guess it
really depends on your usage patterns, but that seems excessive to me-
especially as sstables are promoted.



-- 
Aaron Turner
http://synfin.net/         Twitter: @synfinatic
http://tcpreplay.synfin.net/ - Pcap editing and replay tools for Unix & Windows
Those who would give up essential Liberty, to purchase a little temporary
Safety, deserve neither Liberty nor Safety.
    -- Benjamin Franklin
"carpe diem quam minimum credula postero"

Mime
View raw message