cassandra-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
Subject Re: size tiered compaction - improvement
Date Tue, 03 Apr 2012 21:04:33 GMT
if you know for sure that you will free lot of space compacting some old table, then you can
call UserdefinedCompaction for this table(you can do this from cron). There is also a ticket
in jira with discussion on per-sstable expierd column and tombstones counters.


-----Original Message-----
From: Radim Kolar <>
Sent: Tue, 03 Apr 2012 22:53
Subject: size tiered compaction - improvement

there is problem with size tiered compaction design. It compacts 
together tables of similar size.

sometimes it might happen that you will have some sstables sitting on 
disk forever (Feb 23) because no other similar sized tables were created 
and probably never be. because flushed sstable is about 11-16 mb.

next level about 90 MB
then 5x 90 MB gets compacted to 400 MB sstable
and 5x400 MB ~ 2 GB

problem is that 400 MB sstable is too small to be compacted against 
these 3x 720 MB ones.

-rw-r--r--  1 root  wheel   165M Feb 23 17:03 resultcache-hc-13086-Data.db
-rw-r--r--  1 root  wheel   772M Feb 23 17:04 resultcache-hc-13087-Data.db
-rw-r--r--  1 root  wheel   156M Feb 23 17:06 resultcache-hc-13091-Data.db
-rw-r--r--  1 root  wheel   716M Feb 23 17:18 resultcache-hc-13096-Data.db
-rw-r--r--  1 root  wheel   734M Feb 23 17:29 resultcache-hc-13101-Data.db
-rw-r--r--  1 root  wheel   5.0G Mar 14 09:38 resultcache-hc-13923-Data.db
-rw-r--r--  1 root  wheel   1.9G Mar 16 22:41 resultcache-hc-14084-Data.db
-rw-r--r--  1 root  wheel   1.9G Mar 21 15:11 resultcache-hc-14460-Data.db
-rw-r--r--  1 root  wheel   1.9G Mar 27 05:22 resultcache-hc-14694-Data.db
-rw-r--r--  1 root  wheel   2.0G Mar 31 04:57 resultcache-hc-14851-Data.db
-rw-r--r--  1 root  wheel   112M Mar 31 06:30 resultcache-hc-14922-Data.db
-rw-r--r--  1 root  wheel   577M Apr  1 19:25 resultcache-hc-14943-Data.db

compaction strategy needs to compact sstables by timestamp too. older 
tables should have increased chance to get compacted.
for example - table from today will be compacted with other table in 
range (0.5-1.5) of its size, and this range will get increased with 
sstable age. - 1 month old will have range for example (0.2 - 1.8).

View raw message