incubator-cassandra-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Franc Carter <franc.car...@sirca.org.au>
Subject Re: Large number of files for Leveled Compaction
Date Mon, 17 Jun 2013 04:54:59 GMT
On Mon, Jun 17, 2013 at 2:47 PM, Manoj Mainali <mainalimanoj@gmail.com>wrote:

> With LeveledCompaction, each sstable size is fixed and is defined by
> sstable_size_in_mb in the compaction configuration of CF definition and
> default value is 5MB. In you case, you may have not defined your own value,
> that is why your each sstable is 5MB. And if you dataset is huge, you will
> see a lot of sstable counts.
>


Ok, seems like I do have (at least) an incomplete understanding. I realise
that the minimum size is 5MB, but I thought compaction would merge these
into a smaller number of larger sstables ?

thanks


> Cheers
>
> Manoj
>
>
> On Fri, Jun 7, 2013 at 1:44 PM, Franc Carter <franc.carter@sirca.org.au>wrote:
>
>>
>> Hi,
>>
>> We are trialling Cassandra-1.2(.4) with Leveled compaction as it looks
>> like it may be a win for us.
>>
>> The first step of testing was to push a fairly large slab of data into
>> the Column Family - we did this much faster (> x100) than we would in a
>> production environment. This has left the Column Family with about 140,000
>> files in the Column Family directory which seems way too high. On two of
>> the nodes the CompactionStats show 2 outstanding tasks and on a third node
>> there are over 13,000 outstanding tasks. However from looking at the log
>> activity it looks like compaction has finished on all nodes.
>>
>> Is this number of files expected/normal ?
>>
>> cheers
>>
>> --
>>
>> *Franc Carter* | Systems architect | Sirca Ltd
>>  <marc.zianideferranti@sirca.org.au>
>>
>> franc.carter@sirca.org.au | www.sirca.org.au
>>
>> Tel: +61 2 8355 2514
>>
>> Level 4, 55 Harrington St, The Rocks NSW 2000
>>
>> PO Box H58, Australia Square, Sydney NSW 1215
>>
>>
>>
>


-- 

*Franc Carter* | Systems architect | Sirca Ltd
 <marc.zianideferranti@sirca.org.au>

franc.carter@sirca.org.au | www.sirca.org.au

Tel: +61 2 8355 2514

Level 4, 55 Harrington St, The Rocks NSW 2000

PO Box H58, Australia Square, Sydney NSW 1215

Mime
View raw message