incubator-cassandra-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Steffen Rusitschka <r...@megazebra.com>
Subject Re: Compaction: pending tasks are stuck
Date Wed, 17 Jul 2013 20:48:30 GMT
Hi Aaron, list,

Thanks for the input. But... We figured out what the reason is:

https://issues.apache.org/jira/browse/CASSANDRA-5677

we've eliminated the two major spots where we did deletes and things 
have calmed down. Timeouts still happen and we're eagerly awaiting 1.2.7.

Cheers
Steffen


Am 09.07.2013 00:58, schrieb aaron morton:
>> It started with Cassandra 1.1.10 and on Friday we've migrated to 1.2.6
>> and now we have the issue that flushes are no longer executed
>> automatically - we have to do them every hour by hand (nodetool flush)
>> in order to keep the nodes on an acceptable utilizatzion
>> level. Without manual flush the CPU goes mad after a couple of hours
>> on each instance.
> It may be that because 1.2 manages it's memory better it has to flush to
> disk less frequently in your use case. We flush to disk based on memory
> usage and log file growth.
>
> Can you explain what happens when the CPU goes mad ?
>
>> Is there a way to find out which tasks are pending? Is there a way to
>> cancel them?
> This looks like an error in the way the count is being handled, if there
> are pending tasks they would be running.
> There was a recent issue like this that was fixed in 1.2.5
> https://issues.apache.org/jira/browse/CASSANDRA-5554?attachmentOrder=asc
>
> If this is happening reliability can you create a new ticket and include
> the version and the compaction strategy.
>
> Cheers
>
> -----------------
> Aaron Morton
> Freelance Cassandra Consultant
> New Zealand
>
> @aaronmorton
> http://www.thelastpickle.com
>
> On 8/07/2013, at 9:21 PM, Steffen Rusitschka <rusi@megazebra.com
> <mailto:rusi@megazebra.com>> wrote:
>
>> The heap size is 3GB and the GC has to run only every 30 minutes...
>> I'm pretty sure increasing heap won't help.
>>
>> Am 08.07.2013 11:15, schrieb Radim Kolar:
>>>
>>>> Without manual flush the CPU goes mad after a couple of hours on each
>>>> instance.
>>> increase heap size
>>
>>
>> --
>> Steffen Rusitschka
>> CTO
>> MegaZebra GmbH
>> Steinsdorfstraße 2
>> 81538 München
>> Phone +49 89 80929577
>>
>>
>> rusi@megazebra.com <mailto:rusi@megazebra.com>
>> Challenge me at www.megazebra.com
>>
>> MegaZebra GmbH
>> Geschäftsführer: Henning Kosmack, Christian Meister, Steffen Rusitschka
>> Sitz der Gesellschaft: München, HRB 177947
>


-- 
Steffen Rusitschka
CTO
MegaZebra GmbH
Steinsdorfstraße 2
81538 München
Phone +49 89 80929577


rusi@megazebra.com
Challenge me at www.megazebra.com

MegaZebra GmbH
Geschäftsführer: Henning Kosmack, Christian Meister, Steffen Rusitschka
Sitz der Gesellschaft: München, HRB 177947

Mime
View raw message