incubator-cassandra-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From aaron morton <aa...@thelastpickle.com>
Subject Re: CompactionExecutor holds 8000+ SSTableReader 6G+ memory
Date Sat, 29 Jun 2013 05:09:21 GMT
> Lots of memory are consumed by the SSTableReader's cache
The file cache is managed by the OS. 
However the SSTableReader will have bloom filters and compression meta data, both off heap
in 1.2. The Key and Row caches are global so not associated with any one SStable. 

Cheers
 
-----------------
Aaron Morton
Freelance Cassandra Consultant
New Zealand

@aaronmorton
http://www.thelastpickle.com

On 28/06/2013, at 6:23 PM, sulong <sulong1984@gmail.com> wrote:

> Total 100G data per node.
> 
> 
> On Fri, Jun 28, 2013 at 2:14 PM, sulong <sulong1984@gmail.com> wrote:
> aaron, thanks for your reply. Yes, I do use the Leveled compactions strategy, and the
SSTable size is 10M. If it happens again, I will try to enlarge the sstable size. 
> 
> I just wonder why cassandra doesn't limit the SSTableReader's total memory usage when
compacting. Lots of memory are consumed by the SSTableReader's cache. Why not clear these
cache first at the beginning of compaction?
> 
> 
> On Fri, Jun 28, 2013 at 1:14 PM, aaron morton <aaron@thelastpickle.com> wrote:
> Are you running the Levelled compactions strategy ? 
> If so what is the max SSTable size and what is the total data per node?
> 
>  If you are running it try using a larger SSTable size like 32MB
> 
> Cheers
>  
> -----------------
> Aaron Morton
> Freelance Cassandra Consultant
> New Zealand
> 
> @aaronmorton
> http://www.thelastpickle.com
> 
> On 27/06/2013, at 2:02 PM, sulong <sulong1984@gmail.com> wrote:
> 
>> According to  the OpsCenter records, yes,  the compaction was running then, 8.5mb
/s
>> 
>> 
>> On Thu, Jun 27, 2013 at 9:54 AM, sulong <sulong1984@gmail.com> wrote:
>> version: 1.2.2
>> cluster read requests 800/s, write request 22/s
>> Sorrry, I don't know whether  the compaction was running then.
>> 
>> 
>> On Thu, Jun 27, 2013 at 1:02 AM, Robert Coli <rcoli@eventbrite.com> wrote:
>> On Tue, Jun 25, 2013 at 10:13 PM, sulong <sulong1984@gmail.com> wrote:
>> > I have 4 nodes cassandra cluster. Every node has 32G memory, and the
>> > cassandra jvm uses 8G. The cluster is suffering from gc. Looks like
>> > CompactionExecutor thread holds too many SSTableReader. See the attachement.
>> 
>> What version of Cassandra?
>> What workload?
>> Is compaction actually running?
>> 
>> =Rob
>> 
>> 
> 
> 
> 


Mime
View raw message