tomcat-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "B.V. Prasad" <hi...@yahoo.com>
Subject Re: High CPU with Tomcat 7.0.26 & 6.0.35
Date Tue, 10 Apr 2012 21:14:44 GMT
Th thread dump attached below is when the CPU was showing 55% (even after a day) even after
all the incoming requests have been disabled. I tried with all latest: JDK 6.0.31, APR 1.4.6
& Tomcat native library 1.22. My -Xmx is 2048M, no changes to default PermGen size. I
don't have OOME. The CPU is very high and erratic. Even tried incGc. I am using default settings
for thread pool in server.xml (no changes made).


My server.xml has:

        Connector port="80" protocol="HTTP/1.1" connectionTimeout="20000" redirectPort="443"
        
        Connector port="443" protocol="HTTP/1.1" SSLEnabled="true" maxThreads="100"

    acceptCount="100" scheme="https" secure="true" clientAuth="false" sslProtocol="TLS"

    maxKeepAliveRequests="1" SSLCertificateFile="${catalina.home}/conf/xxx.cert" 
    SSLCertificateKeyFile="${catalina.home}/conf/xxx.key" 
    SSLCertificateChainFile="${catalina.home}/conf/ca.cert"
        
        Connector port="8009" protocol="AJP/1.3" redirectPort="443"

I also set jarscanning to false.

2012-04-06 07:05:52
Full thread dump Java HotSpot(TM) Server VM (10.0-b23 mixed mode):

    "http-80-6" daemon prio=10 tid=0x0a645c00 nid=0x3769 in Object.wait() [0x2fb0b000..0x2fb0be30]
       java.lang.Thread.State: WAITING (on object monitor)
        at java.lang.Object.wait(Native Method)
        - waiting on <0x3ad68cd8> (a org.apache.tomcat.util.net.AprEndpoint$Worker)
        at java.lang.Object.wait(Object.java:485)
        at org.apache.tomcat.util.net.AprEndpoint$Worker.await(AprEndpoint.java:1631)
        - locked <0x3ad68cd8> (a org.apache.tomcat.util.net.AprEndpoint$Worker)
        at org.apache.tomcat.util.net.AprEndpoint$Worker.run(AprEndpoint.java:1656)
        at java.lang.Thread.run(Thread.java:619)
    
    "http-443-227" daemon prio=10 tid=0x0a11a000 nid=0x2a1a in Object.wait() [0x301a7000..0x301a8030]
       java.lang.Thread.State: WAITING (on object monitor)
        at java.lang.Object.wait(Native Method)
        - waiting on <0x3a312c70> (a org.apache.tomcat.util.net.AprEndpoint$Worker)
        at java.lang.Object.wait(Object.java:485)
        at org.apache.tomcat.util.net.AprEndpoint$Worker.await(AprEndpoint.java:1631)
        - locked <0x3a312c70> (a org.apache.tomcat.util.net.AprEndpoint$Worker)
        at org.apache.tomcat.util.net.AprEndpoint$Worker.run(AprEndpoint.java:1656)
        at java.lang.Thread.run(Thread.java:619)
    
    <<< 237 more threads in the same state for both 80 & 443 together >>>
    
    "http-443-205" daemon prio=10 tid=0x096a8400 nid=0x2a04 runnable [0x3089d000..0x3089e130]
       java.lang.Thread.State: RUNNABLE
        at org.apache.tomcat.jni.Socket.sendbb(Native Method)
        at org.apache.coyote.http11.InternalAprOutputBuffer.flushBuffer(InternalAprOutputBuffer.java:711)
        at org.apache.coyote.http11.InternalAprOutputBuffer$SocketOutputBuffer.doWrite(InternalAprOutputBuffer.java:742)
        at org.apache.coyote.http11.filters.IdentityOutputFilter.doWrite(IdentityOutputFilter.java:127)
        at org.apache.coyote.http11.InternalAprOutputBuffer.doWrite(InternalAprOutputBuffer.java:552)
        at org.apache.coyote.Response.doWrite(Response.java:560)
        at org.apache.catalina.connector.OutputBuffer.realWriteBytes(OutputBuffer.java:364)
        at org.apache.tomcat.util.buf.ByteChunk.append(ByteChunk.java:368)
        at org.apache.catalina.connector.OutputBuffer.writeBytes(OutputBuffer.java:392)
        at org.apache.catalina.connector.OutputBuffer.write(OutputBuffer.java:381)
        at org.apache.catalina.connector.CoyoteOutputStream.write(CoyoteOutputStream.java:89)
        at org.apache.catalina.connector.CoyoteOutputStream.write(CoyoteOutputStream.java:83)
        at com.mycompany.xxx.xxx.process(xxx.java:222)
        at com.mycompany.xxx.xxx.doPost(xxx.java:111)
        at javax.servlet.http.HttpServlet.service(HttpServlet.java:637)
        at javax.servlet.http.HttpServlet.service(HttpServlet.java:717)
        at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:290)
        at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
        at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:233)
        at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:191)
        at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127)
        at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102)
        at org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:615)
        at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109)
        at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:293)
        at org.apache.coyote.http11.Http11AprProcessor.process(Http11AprProcessor.java:877)
        at org.apache.coyote.http11.Http11AprProtocol$Http11ConnectionHandler.process(Http11AprProtocol.java:594)
        at org.apache.tomcat.util.net.AprEndpoint$Worker.run(AprEndpoint.java:1675)
        at java.lang.Thread.run(Thread.java:619)
    
    "ajp-8009-Acceptor-0" daemon prio=10 tid=0x333dfc00 nid=0xf54 runnable [0x0494a000..0x0494af30]
       java.lang.Thread.State: RUNNABLE
        at org.apache.tomcat.jni.Socket.accept(Native Method)
        at org.apache.tomcat.util.net.AprEndpoint$Acceptor.run(AprEndpoint.java:1243)
    
    "ajp-8009-CometPoller-0" daemon prio=10 tid=0x334e7c00 nid=0xf53 in Object.wait() [0x048f9000..0x048f9db0]
       java.lang.Thread.State: WAITING (on object monitor)
        at java.lang.Object.wait(Native Method)
        - waiting on <0x384d4378> (a org.apache.tomcat.util.net.AprEndpoint$Poller)
        at java.lang.Object.wait(Object.java:485)
        at org.apache.tomcat.util.net.AprEndpoint$Poller.run(AprEndpoint.java:1426)
        - locked <0x384d4378> (a org.apache.tomcat.util.net.AprEndpoint$Poller)
    
    "ajp-8009-Poller-0" daemon prio=10 tid=0x33729000 nid=0xf52 in Object.wait() [0x048a8000..0x048a8e30]
       java.lang.Thread.State: WAITING (on object monitor)
        at java.lang.Object.wait(Native Method)
        - waiting on <0x38484270> (a org.apache.tomcat.util.net.AprEndpoint$Poller)
        at java.lang.Object.wait(Object.java:485)
        at org.apache.tomcat.util.net.AprEndpoint$Poller.run(AprEndpoint.java:1426)
        - locked <0x38484270> (a org.apache.tomcat.util.net.AprEndpoint$Poller)
    
    "http-443-Acceptor-0" daemon prio=10 tid=0x33728800 nid=0xf4c runnable [0x04713000..0x04713f30]
       java.lang.Thread.State: RUNNABLE
        at org.apache.tomcat.jni.Socket.accept(Native Method)
        at org.apache.tomcat.util.net.AprEndpoint$Acceptor.run(AprEndpoint.java:1243)
    
    "http-443-CometPoller-0" daemon prio=10 tid=0x337bac00 nid=0xf4b in Object.wait() [0x046c2000..0x046c2db0]
       java.lang.Thread.State: WAITING (on object monitor)
        at java.lang.Object.wait(Native Method)
        - waiting on <0x38482310> (a org.apache.tomcat.util.net.AprEndpoint$Poller)
        at java.lang.Object.wait(Object.java:485)
        at org.apache.tomcat.util.net.AprEndpoint$Poller.run(AprEndpoint.java:1426)
        - locked <0x38482310> (a org.apache.tomcat.util.net.AprEndpoint$Poller)
    
    "http-443-Poller-0" daemon prio=10 tid=0x337ba400 nid=0xf48 in Object.wait() [0x0721f000..0x07220130]
       java.lang.Thread.State: WAITING (on object monitor)
        at java.lang.Object.wait(Native Method)
        - waiting on <0x38482178> (a org.apache.tomcat.util.net.AprEndpoint$Poller)
        at java.lang.Object.wait(Object.java:485)
        at org.apache.tomcat.util.net.AprEndpoint$Poller.run(AprEndpoint.java:1426)
        - locked <0x38482178> (a org.apache.tomcat.util.net.AprEndpoint$Poller)
    
    "http-80-Acceptor-0" daemon prio=10 tid=0x3372e400 nid=0xf47 runnable [0x04620000..0x04620fb0]
       java.lang.Thread.State: RUNNABLE
        at org.apache.tomcat.jni.Socket.accept(Native Method)
        at org.apache.tomcat.util.net.AprEndpoint$Acceptor.run(AprEndpoint.java:1243)
    
    "http-80-Sendfile-0" daemon prio=10 tid=0x33531800 nid=0xf46 in Object.wait() [0x045cf000..0x045d0030]
       java.lang.Thread.State: WAITING (on object monitor)
        at java.lang.Object.wait(Native Method)
        - waiting on <0x383905a8> (a org.apache.tomcat.util.net.AprEndpoint$Sendfile)
        at java.lang.Object.wait(Object.java:485)
        at org.apache.tomcat.util.net.AprEndpoint$Sendfile.run(AprEndpoint.java:1900)
        - locked <0x383905a8> (a org.apache.tomcat.util.net.AprEndpoint$Sendfile)
    
    "http-80-CometPoller-0" daemon prio=10 tid=0x335fdc00 nid=0xf45 in Object.wait() [0x0457e000..0x0457eeb0]
       java.lang.Thread.State: WAITING (on object monitor)
        at java.lang.Object.wait(Native Method)
        - waiting on <0x38390408> (a org.apache.tomcat.util.net.AprEndpoint$Poller)
        at java.lang.Object.wait(Object.java:485)
        at org.apache.tomcat.util.net.AprEndpoint$Poller.run(AprEndpoint.java:1426)
        - locked <0x38390408> (a org.apache.tomcat.util.net.AprEndpoint$Poller)
    
    "http-80-Poller-0" daemon prio=10 tid=0x33535000 nid=0xf44 in Object.wait() [0x0452d000..0x0452df30]
       java.lang.Thread.State: WAITING (on object monitor)
        at java.lang.Object.wait(Native Method)
        - waiting on <0x38390270> (a org.apache.tomcat.util.net.AprEndpoint$Poller)
        at java.lang.Object.wait(Object.java:485)
        at org.apache.tomcat.util.net.AprEndpoint$Poller.run(AprEndpoint.java:1426)
        - locked <0x38390270> (a org.apache.tomcat.util.net.AprEndpoint$Poller)
    
    "ContainerBackgroundProcessor[StandardEngine[Catalina]]" daemon prio=10 tid=0x33727c00
nid=0xf43 waiting on condition [0x044dc000..0x044dcdb0]
       java.lang.Thread.State: TIMED_WAITING (sleeping)
        at java.lang.Thread.sleep(Native Method)
        at org.apache.catalina.core.ContainerBase$ContainerBackgroundProcessor.run(ContainerBase.java:1631)
        at java.lang.Thread.run(Thread.java:619)
    
    "Thread-7" daemon prio=10 tid=0x33512400 nid=0xf42 waiting on condition [0x0448b000..0x0448be30]
       java.lang.Thread.State: TIMED_WAITING (sleeping)
        at java.lang.Thread.sleep(Native Method)
        at org.apache.log4j.helpers.FileWatchdog.run(FileWatchdog.java:95)
    
    "Thread-6" daemon prio=10 tid=0x3378b800 nid=0xf41 waiting on condition [0x0443a000..0x0443b0b0]
       java.lang.Thread.State: TIMED_WAITING (sleeping)
        at java.lang.Thread.sleep(Native Method)
        at org.apache.log4j.helpers.FileWatchdog.run(FileWatchdog.java:95)
    
    "Thread-4" daemon prio=10 tid=0x3378c800 nid=0xf3f waiting on condition [0x043e9000..0x043e9fb0]
       java.lang.Thread.State: TIMED_WAITING (sleeping)
        at java.lang.Thread.sleep(Native Method)
        at org.apache.log4j.helpers.FileWatchdog.run(FileWatchdog.java:95)
    
    "Thread-3" daemon prio=10 tid=0x337f7c00 nid=0xf3d waiting on condition [0x00bbb000..0x00bbc030]
       java.lang.Thread.State: TIMED_WAITING (sleeping)
        at java.lang.Thread.sleep(Native Method)
        at org.apache.log4j.helpers.FileWatchdog.run(FileWatchdog.java:95)
    
    "GC Daemon" daemon prio=10 tid=0x3391cc00 nid=0xe8b in Object.wait() [0x00b6a000..0x00b6af30]
       java.lang.Thread.State: TIMED_WAITING (on object monitor)
        at java.lang.Object.wait(Native Method)
        - waiting on <0x37fcec68> (a sun.misc.GC$LatencyLock)
        at sun.misc.GC$Daemon.run(GC.java:100)
        - locked <0x37fcec68> (a sun.misc.GC$LatencyLock)
    
    "Low Memory Detector" daemon prio=10 tid=0x08ece400 nid=0xdd7 runnable [0x00000000..0x00000000]
       java.lang.Thread.State: RUNNABLE
    
    "CompilerThread1" daemon prio=10 tid=0x08eccc00 nid=0xdd6 waiting on condition [0x00000000..0x00668578]
       java.lang.Thread.State: RUNNABLE
    
    "CompilerThread0" daemon prio=10 tid=0x08eca400 nid=0xdd5 waiting on condition [0x00000000..0x005e75f8]
       java.lang.Thread.State: RUNNABLE
    
    "Signal Dispatcher" daemon prio=10 tid=0x08ec9000 nid=0xdd4 runnable [0x00000000..0x00000000]
       java.lang.Thread.State: RUNNABLE
    
    "Finalizer" daemon prio=10 tid=0x08eae400 nid=0xdd3 in Object.wait() [0x0030c000..0x0030d030]
       java.lang.Thread.State: WAITING (on object monitor)
        at java.lang.Object.wait(Native Method)
        - waiting on <0x37f1b410> (a java.lang.ref.ReferenceQueue$Lock)
        at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:116)
        - locked <0x37f1b410> (a java.lang.ref.ReferenceQueue$Lock)
        at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:132)
        at java.lang.ref.Finalizer$FinalizerThread.run(Finalizer.java:159)
    
    "Reference Handler" daemon prio=10 tid=0x08ead400 nid=0xdd2 in Object.wait() [0x002bb000..0x002bbeb0]
       java.lang.Thread.State: WAITING (on object monitor)
        at java.lang.Object.wait(Native Method)
        - waiting on <0x37f1b430> (a java.lang.ref.Reference$Lock)
        at java.lang.Object.wait(Object.java:485)
        at java.lang.ref.Reference$ReferenceHandler.run(Reference.java:116)
        - locked <0x37f1b430> (a java.lang.ref.Reference$Lock)
    
    "main" prio=10 tid=0x08e2a000 nid=0xdce runnable [0x00000000..0xbf7fd7a8]
       java.lang.Thread.State: RUNNABLE
    
    "VM Thread" prio=10 tid=0x08eaa000 nid=0xdd1 runnable 
    
    "GC task thread#0 (ParallelGC)" prio=10 tid=0x08e30c00 nid=0xdcf runnable 
    
    "GC task thread#1 (ParallelGC)" prio=10 tid=0x08e32000 nid=0xdd0 runnable 
    
    "VM Periodic Task Thread" prio=10 tid=0x08ecfc00 nid=0xdd8 waiting on condition 
    
    JNI global references: 811
    
    Heap
     PSYoungGen      total 203456K, used 3336K [0xa9b90000, 0xb7f10000, 0xb7f10000)
      eden space 172928K, 1% used [0xa9b90000,0xa9ed2270,0xb4470000)
      from space 30528K, 0% used [0xb6140000,0xb6140000,0xb7f10000)
      to   space 29504K, 0% used [0xb4470000,0xb4470000,0xb6140000)
     PSOldGen        total 241088K, used 66235K [0x37f10000, 0x46a80000, 0xa9b90000)
      object space 241088K, 27% used [0x37f10000,0x3bfbec28,0x46a80000)
     PSPermGen       total 23168K, used 22999K [0x33f10000, 0x355b0000, 0x37f10000)
      object space 23168K, 99% used [0x33f10000,0x35585e78,0x355b0000)

________________________________
From: Mark Thomas <markt@apache.org>
To: Tomcat Users List <users@tomcat.apache.org> 
Sent: Saturday, March 31, 2012 4:30 AM
Subject: Re: High CPU with Tomcat 7.0.26 & 6.0.35

On 31/03/2012 02:12, B.V. Prasad wrote:
> I am using Tomcat 6.0.16 in production. When I try to upgrade to
> Tomcat 6.0.35, I see high CPU (about 90% vs. about 20% for 6.0.16),
> all other things being equal, except that I had to set
> setBufferSize(16384) in HTTPResponse in order to disable chunked
> encoding (needed as some of my client appliances fail otherwise). I
> see the same high CPU issue even with Tomcat 7.0.26. Any help is
> greatly appreciated.

Thread dumps.

Mark

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org


Mime
View raw message