incubator-cassandra-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Alaa Zubaidi <alaa.zuba...@pdf.com>
Subject Re: 0.7 memory usage problem
Date Fri, 24 Sep 2010 23:12:03 GMT
  My rows consist of only *60 *columns and these 60 columns looks like this:
ColumnName: Sensor59 -- Value: 434.2647915698039 -- TTL: 10800



On 9/24/2010 3:42 PM, Jonathan Ellis wrote:
> looks like you're OOMing trying to compact a very large row. solution:
> smaller rows, or larger heap.
>
> On Fri, Sep 24, 2010 at 3:03 PM, Alaa Zubaidi<alaa.zubaidi@pdf.com>  wrote:
>>   Nothing is working, after disabling swap entirely, the heap is not
>> exhausted but Cassandra crashed with out of memory error.
>> I even slowed down the reading and writing, tried access mode=standard
>> nothing is working.
>> am I using Cassandra wrong or is it Windows?
>>
>> Any help is appreciated?
>> Alaa
>>
>> On 9/21/2010 6:56 PM, Jonathan Ellis wrote:
>>> Disabling swap entirely is usually the easiest fix, yes.
>>>
>>> On Mon, Sep 20, 2010 at 8:10 PM, Alaa Zubaidi<alaa.zubaidi@pdf.com>
>>>   wrote:
>>>>   Thanks Peter,
>>>> I decreased the heap size, it did not help, however, it delayed the
>>>> problem.
>>>> I noticed that its swapping, so, do you think that I should set windows
>>>> to
>>>> Not to swap?
>>>>
>>>> Do you think its related to this issue?
>>>> https://issues.apache.org/jira/browse/CASSANDRA-1014
>>>>
>>>> Thanks,
>>>> Alaa
>>>>
>>>> On 9/18/2010 7:04 AM, Peter Schuller wrote:
>>>>>> I see a spike in heap memory usage on Node 2 where it goes from around
>>>>>> 1G
>>>>>> to
>>>>>> 6GB (max) in less than an hour, and then goes our of memory.
>>>>>> There are some errors in the log file that are reported by other
>>>>>> people,
>>>>>> but
>>>>>> I don't think that these errors are the reason, because it use to
>>>>>> happen
>>>>>> even before I have seen them.
>>>>>>
>>>>>> Can someone help me understand whats going on??
>>>>> Only partially:
>>>>>
>>>>> 54 seconds ParNew GC:s are wild and crazy:
>>>>>
>>>>>     INFO [GC inspection] 2010-09-17 14:53:59,403 GCInspector.java (line
>>>>> 129) GC for ParNew: 54095 ms, 53297952 reclaimed leaving 4712568360
>>>>> used; max is 6563430400
>>>>>
>>>>> Is the machine swapping?
>>>>>
>>>>> I noticed there is hinted hand-off activity going on. Maybe that is a
>>>>> result of nodes dropping in and out due to swapping. In any case, you
>>>>> definitely don't want to have the machine swapping to death. I'm not
>>>>> sure what the best way is to avoid this on Windows, other than
>>>>> decreasing heap size.
>>>>>
>>>>> The repeated exceptions in your log aren't normal as far as I know. I
>>>>> don't remember, but IIRC the UTF-8 encoding issues can be caused by
>>>>> changing partitioner after inserting data (but I'm not sure).
>>>>>
>>>>> With respect to memory use, you don't seem to be inserting so much
>>>>> data for bloom filters and sstable index samples to be a problem.
>>>>> Memtable flushing could cause problems if they're happening too slowly
>>>>> (maybe plausible with swapping) - except that the stage statistics
>>>>> don't indicate there are lots of memtables in memory waiting to be
>>>>> flushed, so that shouldn't be it.
>>>>>
>>>>> Hinted handoff maybe, but I don't remember whether hinted handoff has
>>>>> the potential to accumulate data in RAM (no time to check now).
>>>>>
>>>>> Regardless, I'd recommend fixing any swapping issues you have before
>>>>> trying to draw conclusions about performance. And you don't want those
>>>>> exceptions in your logs.
>>>>>
>>>> --
>>>> Alaa Zubaidi
>>>> PDF Solutions, Inc.
>>>> 333 West San Carlos Street, Suite 700
>>>> San Jose, CA 95110  USA
>>>> Tel: 408-283-5639 (or 408-280-7900 x5639)
>>>> fax: 408-938-6479
>>>> email: alaa.zubaidi@pdf.com
>>>>
>>>>
>>>>
>>>
>> --
>> Alaa Zubaidi
>> PDF Solutions, Inc.
>> 333 West San Carlos Street, Suite 700
>> San Jose, CA 95110  USA
>> Tel: 408-283-5639 (or 408-280-7900 x5639)
>> fax: 408-938-6479
>> email: alaa.zubaidi@pdf.com
>>
>>
>
>

-- 
Alaa Zubaidi
PDF Solutions, Inc.
333 West San Carlos Street, Suite 700
San Jose, CA 95110  USA
Tel: 408-283-5639 (or 408-280-7900 x5639)
fax: 408-938-6479
email: alaa.zubaidi@pdf.com


Mime
View raw message