incubator-cassandra-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Yuki Morishita <mor.y...@gmail.com>
Subject Re: Major compaction does not seems to free the disk space a lot if wide rows are used.
Date Thu, 16 May 2013 14:43:29 GMT
You are right about the behavior of cassandra compaction.
It checks if the key exists on the other SSTable files that are not in
the compaction set.

I think https://issues.apache.org/jira/browse/CASSANDRA-4671 would
help if you upgrade to latest 1.2,
but in your version, I think the workaround is to stop write not to
flush, then compact.

On Thu, May 16, 2013 at 3:07 AM, Boris Yen <yulinyen@gmail.com> wrote:
> Hi All,
>
> Sorry for the wide distribution.
>
> Our cassandra is running on 1.0.10. Recently, we are facing a weird
> situation. We have a column family containing wide rows (each row might
> have a few million of columns). We delete the columns on a daily basis and
> we also run major compaction on it everyday to free up disk space (the
> gc_grace is set to 600 seconds).
>
> However, every time we run the major compaction, only 1 or 2GB disk space
> is freed. We tried to delete most of the data before running compaction,
> however, the result is pretty much the same.
>
> So, we tried to check the source code. It seems that the column tombstones
> could only be purged when the row key is not in other sstables. I know the
> major compaction should include all sstables, however, in our use case,
> columns get inserted rapidly. This will make the cassandra flush the
> memtables to disk and create new sstables. The newly created sstables will
> have the same keys as the sstables that are being compacted (the compaction
> will take 2 or 3 hours to finish). My question is that will these newly
> created sstables be the cause of why most of the column-tombstone not being
> purged?
>
> p.s. We also did some other tests. We inserted data to the same CF with the
> same wide-row pattern and deleted most of the data. This time we stopped
> all the writes to cassandra and did the compaction. The disk usage
> decreased dramatically.
>
> Any suggestions or is this a know issue.
>
> Thanks and Regards,
> Boris



-- 
Yuki Morishita
 t:yukim (http://twitter.com/yukim)

Mime
View raw message