tomcat-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Leon Rosenberg <rosenberg.l...@googlemail.com>
Subject Re: Tomcat 100% CPU usage after moving from Java 5 to 6
Date Mon, 25 Jan 2010 11:33:45 GMT
P.S. Your tomcat is writing gc logs, please attach them next time.

On Mon, Jan 25, 2010 at 12:31 PM, Leon Rosenberg
<rosenberg.leon@googlemail.com> wrote:
> take a look on threads
> ajp-8009-2
> ajp-8009-18
> ajp-8009-30
> ajp-8009-38
>
> they are all seem to be either in an infinite loop:
>        at java.util.AbstractList$Itr.hasNext(AbstractList.java:339)
>        at nl.indicia.vip.framework.util.Workflow.performWorkflowAction(Workflow.java:166)
>        at nl.indicia.vip.framework.util.DQWorkflow.performWorkflowAction(DQWorkflow.java:56)
>        at org.apache.jsp.bamplaza.componenten.workflow.xmlworkflow_jsp._jspService(xmlworkflow_jsp.java:382)
>
> or iterating over a very huge list. This way or other they are
> consuming your cpu:
> "ajp-8009-18" Id=187 in RUNNABLE cpu=236296 ms usr=236265 ms blocked
> 315 for -1 ms waited 256 for -1 ms
> "ajp-8009-2" Id=171 in RUNNABLE cpu=221968 ms usr=218734 ms blocked
> 12822 for -1 ms waited 10071 for -1 ms
> "ajp-8009-30" Id=199 in RUNNABLE cpu=371562 ms usr=368234 ms blocked
> 9527 for -1 ms waited 7543 for -1 ms
> "ajp-8009-38" Id=210 in RUNNABLE cpu=518890 ms usr=515593 ms blocked
> 10352 for -1 ms waited 9035 for -1 ms
>
> So either its a bug (maybe a concurrency issue, since java 1.6. has
> improved a lot in terms of lock contention reduction) or,
> its a side effect of an out of memory problem, can you make a heap
> dump with jmap and check whether your Old Gen is 99.9999% full?
>
> Otherwise you should look into at
> nl.indicia.vip.framework.util.Workflow.performWorkflowAction(Workflow.java:166)
> which I never heard of, and don't have sources to check, what's
> happening there exactly.
>
> Good luck
> Leon
>
>
>
>
>
>
> On Mon, Jan 25, 2010 at 12:14 PM, Pid <pid@pidster.com> wrote:
>> On 25/01/2010 11:08, Jesse Klaasse wrote:
>>>
>>> I am running a production environment for a website (over half a million
>>> hits per day), using Tomcat 5.5.20 (I'm stuck to that version due to
>>> support restrictions) behind IIS 6 using JK connector 1.2.28,
>>> tcnative-1.dll (1.1.19) on Windows Server 2003 R2 Enterprise x64 SP2
>>> (with 16 GB RAM). Until last week, I have been using Sun Java 5. Last
>>> week I have upgraded the JDK to Sun Java 6.
>>> The system uses a MS SQL Server 2005 database, which resides on another
>>> server.
>>> That's when things went wrong. At first, all seemed fine (average CPU
>>> usage seems lower than when using Java 5), but after an hour or so (30
>>> minutes - 1,5 hour max) after a Tomcat restart, Tomcat's CPU usage jumps
>>> to 100% (average CPU usage using Java 5 is around 30-40%, peaks around
>>> 60%). I can't really figure out what is causing this. I have retried
>>> about 6 restarts (including a complete system reboot), but without any
>>> success. When I move back to Java 5, the problems are gone.
>>> My Tomcat Java arguments:
>>> -Dcatalina.base=D:\tomcat -Dcatalina.home=D:\tomcat
>>> -Djava.endorsed.dirs=D:\tomcat\common\endorsed
>>> -Djava.io.tmpdir=D:\tomcat\temp
>>>
>>> -Djavax.sql.DataSource.Factory=org.apache.commons.dbcp.BasicDataSourceFactory
>>> -Dcom.sun.management.jmxremote -XX:MaxPermSize=512m
>>> -Xloggc:D:\logs\gc\tomcat-gc.log -XX:+PrintGCDetails -XX:+UseParNewGC
>>> -XX:+UseConcMarkSweepGC -XX:+UseTLAB -XX:+CMSClassUnloadingEnabled
>>> -XX:+CMSPermGenSweepingEnabled -Xms4096m -Xmx10240m
>>> Some further information which could be important:
>>> - I have tried to reproduce the problems on my development environment
>>> (using jMeter for generating server load), but without success;
>>> - I have created some thread dumps during the 100% CPU periods (attached
>>> as a zip file);
>>> I would really like to make the move to Java 6, since Java 5's is EOL.
>>> Any ideas, anyone?
>>
>> Have you tested those memory and garbage collection settings with this
>> version of Java?
>>
>> What does the garbage collection log indicate, normal or abnormal activity?
>>
>> Java 6 has some pretty good tools for examining a running JVM, have a nose
>> around in the java/bin directory.
>>
>>
>> p
>>
>>
>>
>>
>>
>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
>>> For additional commands, e-mail: users-help@tomcat.apache.org
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
>> For additional commands, e-mail: users-help@tomcat.apache.org
>>
>>
>

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org


Mime
View raw message