cassandra-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Nick Bailey <n...@riptano.com>
Subject Re: When to call the major compaction ?
Date Wed, 01 Dec 2010 14:49:48 GMT
The part about gc refers to old sstable files on disk. After a compaction,
the old files on disk will be deleted when garbage collection happens.

On Wed, Dec 1, 2010 at 7:31 AM, Ying Tang <ivytang0812@gmail.com> wrote:

> I'm confused , plz ingore the mail above.
> Here is my confusion ,
>    posterior to 0.6.6/0.7  , minor compaction and major compaction both
> can clean out rows 'tagged'  tombstones  , and generate a new , without
> tombstones , sstable .
>     And the tombstones remains in memory ,waiting to be removed by jvm gc .
> Am i right?
>
> On Wed, Dec 1, 2010 at 9:10 PM, Ying Tang <ivytang0812@gmail.com> wrote:
>
>> 1. So posterior to 0.6.6/0.7 ,  minor compaction and major compaction
>> both  can clean out rows 'tagged'  tombstones , this kind of clean out
>> doesn't mead remove it from the disk permanently.
>>     The real remove is done by the jvm GC ?
>> 2. The intence of compaction is merging multi sstables into one , clean
>> out the tombstone , let the un-tombstones  rows be into a new ordered
>> sstable ?
>>
>>
>>
>> On Wed, Dec 1, 2010 at 7:30 PM, Sylvain Lebresne <sylvain@yakaz.com>wrote:
>>
>>> On Wed, Dec 1, 2010 at 12:11 PM, Ying Tang <ivytang0812@gmail.com>
>>> wrote:
>>> > And i have another question , what's the difference between minor
>>> > compaction and major compaction?
>>>
>>> A major compaction is a compaction that compact *all* the SSTables of a
>>> given
>>> column family (compaction compacts one CF at a time).
>>>
>>> Before https://issues.apache.org/jira/browse/CASSANDRA-1074
>>> (introduced in 0.6.6 and
>>> recent 0.7 betas/rcs), major compactions where the only ones that removed
>>> the
>>> tombstones (see http://wiki.apache.org/cassandra/DistributedDeletes)
>>> and this is the
>>> reason major compaction exists. Now, with #1074, minor compactions
>>> should remove most
>>> if not all tombstones, so major compaction are not or much less useful
>>> (it may depend on your
>>> workload though as minor can't always delete the tombstones).
>>>
>>> --
>>> Sylvain
>>>
>>> >
>>> > On 12/1/10, Chen Xinli <chen.daqi@gmail.com> wrote:
>>> >> 2010/12/1 Ying Tang <ivytang0812@gmail.com>
>>> >>
>>> >>> Every time cassandra creates a new sstable , it will call the
>>> >>> CompactionManager.submitMinorIfNeeded  ? And if the number of
>>> memtables is
>>> >>> beyond  MinimumCompactionThreshold  , the minor compaction will
be
>>> called.
>>> >>> And there is also a method named CompactionManager.submitMajor ,
and
>>> the
>>> >>> call relationship is :
>>> >>>
>>> >>> NodeCmd -- > NodeProbe -->StorageService.forceTableCompaction
-->
>>> >>> Table.forceCompaction -->CompactionManager.performMajor -->
>>> >>> CompactionManager.submitMajor
>>> >>>
>>> >>> ColumnFamilyStore.forceMajorCompaction -->
>>> CompactionManager.performMajor
>>> >>> --> CompactionManager.submitMajor
>>> >>>
>>> >>>
>>> >>> HintedHandOffManager
>>> >>>  --> CompactionManager.submitMajor
>>> >>>
>>> >>> So i have 3 questions:
>>> >>> 1. Once a new sstable has been created ,
>>> >>> CompactionManager.submitMinorIfNeeded  will be called ,
>>> minorCompaction
>>> >>> maybe called .
>>> >>>     But when will the majorCompaction be called ? Just the NodeCmd
?
>>> >>>
>>> >>
>>> >> Yes, majorCompaction must be called manually from NodeCmd
>>> >>
>>> >>
>>> >>> 2. Which jobs will minorCompaction and majorCompaction do ?
>>> >>>     Will minorCompaction delete the data that have been marked as
>>> deleted
>>> >>> ?
>>> >>>     And how about the major compaction ?
>>> >>>
>>> >>
>>> >> Compaction only mark sstables as deleted. Deletion will be done when
>>> there
>>> >> are full gc, or node restarted.
>>> >>
>>> >>
>>> >>> 3. When gc be called ? Every time compaction been called?
>>> >>>
>>> >>
>>> >> GC has nothing to do with compaction, you may mistake the two
>>> conceptions
>>> >>
>>> >>
>>> >>>
>>> >>>
>>> >>>
>>> >>> --
>>> >>> Best regards,
>>> >>>
>>> >>> Ivy Tang
>>> >>>
>>> >>>
>>> >>>
>>> >>>
>>> >>
>>> >>
>>> >> --
>>> >> Best Regards,
>>> >> Chen Xinli
>>> >>
>>> >
>>> >
>>> > --
>>> > Best regards,
>>> >
>>> > Ivy Tang
>>> >
>>>
>>
>>
>>
>> --
>> Best regards,
>>
>> Ivy Tang
>>
>>
>>
>>
>
>
> --
> Best regards,
>
> Ivy Tang
>
>
>
>

Mime
View raw message