cassandra-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ying Tang <ivytang0...@gmail.com>
Subject Re: When to call the major compaction ?
Date Wed, 01 Dec 2010 13:31:14 GMT
I'm confused , plz ingore the mail above.
Here is my confusion ,
   posterior to 0.6.6/0.7  , minor compaction and major compaction both  can
clean out rows 'tagged'  tombstones  , and generate a new , without
tombstones , sstable .
    And the tombstones remains in memory ,waiting to be removed by jvm gc .
Am i right?

On Wed, Dec 1, 2010 at 9:10 PM, Ying Tang <ivytang0812@gmail.com> wrote:

> 1. So posterior to 0.6.6/0.7 ,  minor compaction and major compaction both
> can clean out rows 'tagged'  tombstones , this kind of clean out doesn't
> mead remove it from the disk permanently.
>     The real remove is done by the jvm GC ?
> 2. The intence of compaction is merging multi sstables into one , clean out
> the tombstone , let the un-tombstones  rows be into a new ordered sstable ?
>
>
>
> On Wed, Dec 1, 2010 at 7:30 PM, Sylvain Lebresne <sylvain@yakaz.com>wrote:
>
>> On Wed, Dec 1, 2010 at 12:11 PM, Ying Tang <ivytang0812@gmail.com> wrote:
>> > And i have another question , what's the difference between minor
>> > compaction and major compaction?
>>
>> A major compaction is a compaction that compact *all* the SSTables of a
>> given
>> column family (compaction compacts one CF at a time).
>>
>> Before https://issues.apache.org/jira/browse/CASSANDRA-1074
>> (introduced in 0.6.6 and
>> recent 0.7 betas/rcs), major compactions where the only ones that removed
>> the
>> tombstones (see http://wiki.apache.org/cassandra/DistributedDeletes)
>> and this is the
>> reason major compaction exists. Now, with #1074, minor compactions
>> should remove most
>> if not all tombstones, so major compaction are not or much less useful
>> (it may depend on your
>> workload though as minor can't always delete the tombstones).
>>
>> --
>> Sylvain
>>
>> >
>> > On 12/1/10, Chen Xinli <chen.daqi@gmail.com> wrote:
>> >> 2010/12/1 Ying Tang <ivytang0812@gmail.com>
>> >>
>> >>> Every time cassandra creates a new sstable , it will call the
>> >>> CompactionManager.submitMinorIfNeeded  ? And if the number of
>> memtables is
>> >>> beyond  MinimumCompactionThreshold  , the minor compaction will be
>> called.
>> >>> And there is also a method named CompactionManager.submitMajor , and
>> the
>> >>> call relationship is :
>> >>>
>> >>> NodeCmd -- > NodeProbe -->StorageService.forceTableCompaction
-->
>> >>> Table.forceCompaction -->CompactionManager.performMajor -->
>> >>> CompactionManager.submitMajor
>> >>>
>> >>> ColumnFamilyStore.forceMajorCompaction -->
>> CompactionManager.performMajor
>> >>> --> CompactionManager.submitMajor
>> >>>
>> >>>
>> >>> HintedHandOffManager
>> >>>  --> CompactionManager.submitMajor
>> >>>
>> >>> So i have 3 questions:
>> >>> 1. Once a new sstable has been created ,
>> >>> CompactionManager.submitMinorIfNeeded  will be called ,
>> minorCompaction
>> >>> maybe called .
>> >>>     But when will the majorCompaction be called ? Just the NodeCmd ?
>> >>>
>> >>
>> >> Yes, majorCompaction must be called manually from NodeCmd
>> >>
>> >>
>> >>> 2. Which jobs will minorCompaction and majorCompaction do ?
>> >>>     Will minorCompaction delete the data that have been marked as
>> deleted
>> >>> ?
>> >>>     And how about the major compaction ?
>> >>>
>> >>
>> >> Compaction only mark sstables as deleted. Deletion will be done when
>> there
>> >> are full gc, or node restarted.
>> >>
>> >>
>> >>> 3. When gc be called ? Every time compaction been called?
>> >>>
>> >>
>> >> GC has nothing to do with compaction, you may mistake the two
>> conceptions
>> >>
>> >>
>> >>>
>> >>>
>> >>>
>> >>> --
>> >>> Best regards,
>> >>>
>> >>> Ivy Tang
>> >>>
>> >>>
>> >>>
>> >>>
>> >>
>> >>
>> >> --
>> >> Best Regards,
>> >> Chen Xinli
>> >>
>> >
>> >
>> > --
>> > Best regards,
>> >
>> > Ivy Tang
>> >
>>
>
>
>
> --
> Best regards,
>
> Ivy Tang
>
>
>
>


-- 
Best regards,

Ivy Tang

Mime
View raw message