incubator-cassandra-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Sylvain Lebresne <sylv...@yakaz.com>
Subject Re: When to call the major compaction ?
Date Wed, 01 Dec 2010 11:30:52 GMT
On Wed, Dec 1, 2010 at 12:11 PM, Ying Tang <ivytang0812@gmail.com> wrote:
> And i have another question , what's the difference between minor
> compaction and major compaction?

A major compaction is a compaction that compact *all* the SSTables of a given
column family (compaction compacts one CF at a time).

Before https://issues.apache.org/jira/browse/CASSANDRA-1074
(introduced in 0.6.6 and
recent 0.7 betas/rcs), major compactions where the only ones that removed the
tombstones (see http://wiki.apache.org/cassandra/DistributedDeletes)
and this is the
reason major compaction exists. Now, with #1074, minor compactions
should remove most
if not all tombstones, so major compaction are not or much less useful
(it may depend on your
workload though as minor can't always delete the tombstones).

--
Sylvain

>
> On 12/1/10, Chen Xinli <chen.daqi@gmail.com> wrote:
>> 2010/12/1 Ying Tang <ivytang0812@gmail.com>
>>
>>> Every time cassandra creates a new sstable , it will call the
>>> CompactionManager.submitMinorIfNeeded  ? And if the number of memtables is
>>> beyond  MinimumCompactionThreshold  , the minor compaction will be called.
>>> And there is also a method named CompactionManager.submitMajor , and the
>>> call relationship is :
>>>
>>> NodeCmd -- > NodeProbe -->StorageService.forceTableCompaction -->
>>> Table.forceCompaction -->CompactionManager.performMajor -->
>>> CompactionManager.submitMajor
>>>
>>> ColumnFamilyStore.forceMajorCompaction --> CompactionManager.performMajor
>>> --> CompactionManager.submitMajor
>>>
>>>
>>> HintedHandOffManager
>>>  --> CompactionManager.submitMajor
>>>
>>> So i have 3 questions:
>>> 1. Once a new sstable has been created ,
>>> CompactionManager.submitMinorIfNeeded  will be called , minorCompaction
>>> maybe called .
>>>     But when will the majorCompaction be called ? Just the NodeCmd ?
>>>
>>
>> Yes, majorCompaction must be called manually from NodeCmd
>>
>>
>>> 2. Which jobs will minorCompaction and majorCompaction do ?
>>>     Will minorCompaction delete the data that have been marked as deleted
>>> ?
>>>     And how about the major compaction ?
>>>
>>
>> Compaction only mark sstables as deleted. Deletion will be done when there
>> are full gc, or node restarted.
>>
>>
>>> 3. When gc be called ? Every time compaction been called?
>>>
>>
>> GC has nothing to do with compaction, you may mistake the two conceptions
>>
>>
>>>
>>>
>>>
>>> --
>>> Best regards,
>>>
>>> Ivy Tang
>>>
>>>
>>>
>>>
>>
>>
>> --
>> Best Regards,
>> Chen Xinli
>>
>
>
> --
> Best regards,
>
> Ivy Tang
>

Mime
View raw message