tomcat-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Edson Richter <edsonrich...@hotmail.com>
Subject Re: Help in diagnosing server unresponsiveness
Date Wed, 06 Feb 2013 11:12:33 GMT
Em 06/02/2013 01:18, Zoran Avtarovski escreveu:
> Thanks Igor,
>
> I just stumbled upon that same document. I think you may be on to
> something here.
>
> I have a feeling that the GC may not be configured well.

Also read the following:
http://docs.oracle.com/cd/E13150_01/jrockit_jvm/jrockit/geninfo/diagnos/bestpractices.html


And check this (if you are using RHEL 6 or CentOS 6, worth to take a look):
https://rhn.redhat.com/errata/RHSA-2013-0223.html


Edson

>
> Z.
>
>
> On 6/02/13 2:15 PM, "Igor Cicimov" <icicimov@gmail.com> wrote:
>
>> On Wed, Feb 6, 2013 at 1:15 PM, Zoran Avtarovski
>> <zoran@sparecreative.com>wrote:
>>
>>> Here's some updated observations after a not quite incident (CPU and
>>> memory spiked but the app is still running):
>>>
>>> 1. Yesterday we had a 90% CPU spike at a time where there was absolutely
>>> no server traffic. Verified through both the HTTP logs and the mod_jk
>>> logs. The CPU spiked and recovered back to average levels.
>>> 2. Used memory spiked at 10GB from a pre incident average of 500MB
>>> throughout 2 busy days without incident
>>> 3. Used memory has only gone back down to 4GB and is holding at this
>>> level
>>> 4. The Used physical memory went up from 2GB to 14GB and has stayed
>>> there
>>> 5. Garbage collector time spikes to 24.0. I think with JavaMelody it
>>> means
>>> that GC took 24% of  of the CPU??
>>>
>>> So I think our issues are related to GC. Is there a way to trigger more
>>> frequent GC which will hopefully be less resource intensive?
>>>
>>> And why have the memory usage levels not recovered?
>>>
>>> Z.
>>>
>>>
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
>>> For additional commands, e-mail: users-help@tomcat.apache.org
>>>
>>>
>> Zoran,
>>
>> First I would like to recommend the following document for reading:
>> http://www.oracle.com/technetwork/java/javase/gc-tuning-6-140523.html#cms
>>
>> It explains the GC in JVM 1.6 including the Concurrent Collector settings
>> which is the one you are using. The values for the GC in the log file are
>> the time the particular collector spent for the operation so the 24.0
>> would
>> probably mean 24 seconds, which might mean that for 24 seconds your
>> applications might be unresponsive during the CMS GC.
>>
>> As explained in the document from the above link the GC has minor and
>> major
>> collecting phases. During the minor collection the objects are GC'ed from
>> the so called young generation or promoted in the old (tenured)
>> generation.
>> More of this objects pile up in the old generation more frequent the major
>> GC needs to run. The major ones take usually much longer (they have to
>> clean much bigger space) time than the minor ones but the minor ones have
>> to run more frequently. Now, what you need to find is what is causing your
>> problem really? If your application creates lot of new objects then you'll
>> have lots of minor GC running. More of this objects survive, they get
>> moved
>> to the old generation space and then you'll have lots of major GC running
>> as well. The danger here is that you might end up with constantly running
>> GC which will render your application unusable due to pauses. So basically
>> badly written application can cause lots of problems, not closing
>> connections and freeing objects etc etc, and in that case even the best GC
>> tunning in the world will not help you, your application(s) will
>> eventually
>> get to halt.
>>
>> So read the document carefully and decide which user case is best for you.
>> If you are creating lots of new objects then maybe increasing the minor
>> space (default new/old ratio is 1:3) can help.
>>
>> Also paste here the results of the GC logs. The link I provided has some
>> more useful settings and recommendations for the CMS collector. This
>> collector stops the application threads twice during the operation so you
>> need to check those times too.
>>
>> Cheers & Pozdrav,
>> Igor
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
> For additional commands, e-mail: users-help@tomcat.apache.org
>
>
>


---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org


Mime
View raw message