cassandra-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Redmumba <redmu...@gmail.com>
Subject Re: Why does recompacting a table with no changes or tombstones result in work?
Date Tue, 21 Oct 2014 19:23:29 GMT
Much appreciated, Robert.  I was under the (unfounded) impression that the
tombstones and similar were stored in the index or other auxilliary file.
A false assumption, to be sure.

For readers: To support this specific use case, my plan is to have a job
that simply parses out the "sstable ID" from the table's files to see if
there's more than one for a given table.  If there are, it means there are
multiple sstables that can be compacted.

On Mon, Oct 20, 2014 at 12:03 PM, Robert Coli <rcoli@eventbrite.com> wrote:

> On Mon, Oct 20, 2014 at 10:24 AM, Redmumba <redmumba@gmail.com> wrote:
>
>> I ran into an interesting issue--when I run compaction on a table that is
>> already compacted, it still, well... compacts.  The table's TTL is set to
>> 0, there are no deletes or other writes to these tables, and I confirmed
>> (on disk) that there was only a single set of files.  There are no changes
>> pending in memory for this table, either.
>>
>> Why is compaction being performed on a table that has no changes?  Are
>> there other reasons for compaction to be re-run, or does the compact
>> command to nodetool just blindly do what it's told?
>>
>
> The latter. The compaction process doesn't "know" that the file is
> compacted and it doesn't "know" that there have been no writes since it was
> compacted. It also doesn't know that the file doesn't contain TTLed columns
> or deletes with timestamps in the future from write time at first
> compaction, but which a second compaction would turn into tombstones. Or
> tombstones which would be removed after gc_grace_seconds, etc...
>
> In trunk Cassandra, SSTables are marked "repaired" which would prevent
> them from repairing. In theory maybe (?) a similar technique be used with
> regards to compaction, but Cassandra dev team quite rationally does not
> optimize for the case where the dataset is static...
>
> =Rob
>
>

Mime
View raw message