incubator-cassandra-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Alaa Zubaidi <alaa.zuba...@pdf.com>
Subject Re: 0.7 memory usage problem
Date Tue, 21 Sep 2010 01:10:21 GMT
  Thanks Peter,
I decreased the heap size, it did not help, however, it delayed the problem.
I noticed that its swapping, so, do you think that I should set windows 
to Not to swap?

Do you think its related to this issue?
https://issues.apache.org/jira/browse/CASSANDRA-1014

Thanks,
Alaa

On 9/18/2010 7:04 AM, Peter Schuller wrote:
>> I see a spike in heap memory usage on Node 2 where it goes from around 1G to
>> 6GB (max) in less than an hour, and then goes our of memory.
>> There are some errors in the log file that are reported by other people, but
>> I don't think that these errors are the reason, because it use to happen
>> even before I have seen them.
>>
>> Can someone help me understand whats going on??
> Only partially:
>
> 54 seconds ParNew GC:s are wild and crazy:
>
>     INFO [GC inspection] 2010-09-17 14:53:59,403 GCInspector.java (line
> 129) GC for ParNew: 54095 ms, 53297952 reclaimed leaving 4712568360
> used; max is 6563430400
>
> Is the machine swapping?
>
> I noticed there is hinted hand-off activity going on. Maybe that is a
> result of nodes dropping in and out due to swapping. In any case, you
> definitely don't want to have the machine swapping to death. I'm not
> sure what the best way is to avoid this on Windows, other than
> decreasing heap size.
>
> The repeated exceptions in your log aren't normal as far as I know. I
> don't remember, but IIRC the UTF-8 encoding issues can be caused by
> changing partitioner after inserting data (but I'm not sure).
>
> With respect to memory use, you don't seem to be inserting so much
> data for bloom filters and sstable index samples to be a problem.
> Memtable flushing could cause problems if they're happening too slowly
> (maybe plausible with swapping) - except that the stage statistics
> don't indicate there are lots of memtables in memory waiting to be
> flushed, so that shouldn't be it.
>
> Hinted handoff maybe, but I don't remember whether hinted handoff has
> the potential to accumulate data in RAM (no time to check now).
>
> Regardless, I'd recommend fixing any swapping issues you have before
> trying to draw conclusions about performance. And you don't want those
> exceptions in your logs.
>

-- 
Alaa Zubaidi
PDF Solutions, Inc.
333 West San Carlos Street, Suite 700
San Jose, CA 95110  USA
Tel: 408-283-5639 (or 408-280-7900 x5639)
fax: 408-938-6479
email: alaa.zubaidi@pdf.com



Mime
View raw message