tomcat-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From David kerber <dcker...@verizon.net>
Subject Re: Performance with many small requests
Date Mon, 11 May 2009 13:42:00 GMT
Peter Crowther wrote:
>> From: David kerber [dckerber@verizon.net]
>> My cpu usage for tomcat
>> has gone from bouncing between 0 and 1 in task manager, to a steady 2
>> since more threads are now actually doing work instead of waiting around
>> for their turn at the code, my disk writes per sec in perfmon have also
>> more than doubled, and the destination log file is growing much faster
>> as well.
>>     
>
> All excellent news.  The fact that you've seen the performance double means that there
was, in fact, a bottleneck there.  Have you taken a new thread dump to see whether the locks
(almost certainly on the log write) are still a problem?  If so, you might have to go to a
more complex scheme such as multiple log files managed by a pool manager.  Don't even try
to write the pool manager yourself; they're horribly messy things to get right and shake the
race conditions out*.  I half-remember Jakarta Commons has one that can be adapted if you
get to that stage.
>   
 From what I can tell now, it looks like most of my wait time is on 
socket reads.  In the thread dump I took about 20 minutes ago, I didn't 
see any waiting on disk writes:

The line listed in this one is my inputStream.read():

[2009-05-11 08:20:09] [info] "http-1024-Processor8"
[2009-05-11 08:20:09] [info] daemon
[2009-05-11 08:20:09] [info] prio=6 tid=0x270e83c8
[2009-05-11 08:20:09] [info] nid=0xcd4
[2009-05-11 08:20:09] [info] runnable
[2009-05-11 08:20:09] [info] [0x2755f000..0x2755f9e4]
[2009-05-11 08:20:09] [info]     at 
java.net.SocketInputStream.socketRead0(Native Method)
[2009-05-11 08:20:10] [info]     at 
java.net.SocketInputStream.read(Unknown Source)
[2009-05-11 08:20:10] [info]     at 
org.apache.coyote.http11.InternalInputBuffer.fill(InternalInputBuffer.java:747)
[2009-05-11 08:20:10] [info]     at 
org.apache.coyote.http11.InternalInputBuffer$InputStreamInputBuffer.doRead(InternalInputBuffer.java:777)
[2009-05-11 08:20:10] [info]     at 
org.apache.coyote.http11.filters.IdentityInputFilter.doRead(IdentityInputFilter.java:115)
[2009-05-11 08:20:10] [info]     at 
org.apache.coyote.http11.InternalInputBuffer.doRead(InternalInputBuffer.java:712)
[2009-05-11 08:20:10] [info]     at 
org.apache.coyote.Request.doRead(Request.java:423)
[2009-05-11 08:20:10] [info]     at 
org.apache.catalina.connector.InputBuffer.realReadBytes(InputBuffer.java:283)
[2009-05-11 08:20:10] [info]     at 
org.apache.tomcat.util.buf.ByteChunk.substract(ByteChunk.java:404)
[2009-05-11 08:20:10] [info]     at 
org.apache.catalina.connector.InputBuffer.read(InputBuffer.java:298)
[2009-05-11 08:20:10] [info]     at 
org.apache.catalina.connector.CoyoteInputStream.read(CoyoteInputStream.java:192)
[2009-05-11 08:20:10] [info]     at 
eddsrv.EddRcvr.processRequest(EddRcvr.java:199)
[2009-05-11 08:20:10] [info]     at eddsrv.EddRcvr.doPost(EddRcvr.java:94)
[2009-05-11 08:20:10] [info]     at 
javax.servlet.http.HttpServlet.service(HttpServlet.java:709)
[2009-05-11 08:20:10] [info]     at 
javax.servlet.http.HttpServlet.service(HttpServlet.java:802)
[2009-05-11 08:20:10] [info]     at 
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:252)
[2009-05-11 08:20:10] [info]     at 
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:173)
[2009-05-11 08:20:10] [info]     at 
org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:213)
[2009-05-11 08:20:10] [info]     at 
org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:178)
[2009-05-11 08:20:10] [info]     at 
org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:126)
[2009-05-11 08:20:10] [info]     at 
org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:105)
[2009-05-11 08:20:10] [info]     at 
org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:107)
[2009-05-11 08:20:11] [info]     at 
org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:148)
[2009-05-11 08:20:11] [info]     at 
org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:869)
[2009-05-11 08:20:11] [info]     at 
org.apache.coyote.http11.Http11BaseProtocol$Http11ConnectionHandler.processConnection(Http11BaseProtocol.java:667)
[2009-05-11 08:20:11] [info]     at 
org.apache.tomcat.util.net.PoolTcpEndpoint.processSocket(PoolTcpEndpoint.java:527)
[2009-05-11 08:20:11] [info]     at 
org.apache.tomcat.util.net.LeaderFollowerWorkerThread.runIt(LeaderFollowerWorkerThread.java:80)
[2009-05-11 08:20:11] [info]     at 
org.apache.tomcat.util.threads.ThreadPool$ControlRunnable.run(ThreadPool.java:684)
[2009-05-11 08:20:11] [info]     at java.lang.Thread.run(Unknown Source)
[2009-05-11 08:20:11] [info]


This one seems to be waiting on something in tomcat itself:

[2009-05-11 08:19:49] [info] "http-1024-Processor45"
[2009-05-11 08:19:49] [info] daemon
[2009-05-11 08:19:49] [info] prio=6 tid=0x26fa6f38
[2009-05-11 08:19:49] [info] nid=0x340
[2009-05-11 08:19:49] [info] runnable
[2009-05-11 08:19:49] [info] [0x27edf000..0x27edfae4]
[2009-05-11 08:19:49] [info]     at 
java.net.SocketInputStream.socketRead0(Native Method)
[2009-05-11 08:19:49] [info]     at 
java.net.SocketInputStream.read(Unknown Source)
[2009-05-11 08:19:49] [info]     at 
org.apache.coyote.http11.InternalInputBuffer.fill(InternalInputBuffer.java:737)
[2009-05-11 08:19:49] [info]     at 
org.apache.coyote.http11.InternalInputBuffer.parseRequestLine(InternalInputBuffer.java:398)
[2009-05-11 08:19:50] [info]     at 
org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:827)
[2009-05-11 08:19:50] [info]     at 
org.apache.coyote.http11.Http11BaseProtocol$Http11ConnectionHandler.processConnection(Http11BaseProtocol.java:667)
[2009-05-11 08:19:50] [info]     at 
org.apache.tomcat.util.net.PoolTcpEndpoint.processSocket(PoolTcpEndpoint.java:527)
[2009-05-11 08:19:50] [info]     at 
org.apache.tomcat.util.net.LeaderFollowerWorkerThread.runIt(LeaderFollowerWorkerThread.java:80)
[2009-05-11 08:19:50] [info]     at 
org.apache.tomcat.util.threads.ThreadPool$ControlRunnable.run(ThreadPool.java:684)
[2009-05-11 08:19:50] [info]     at java.lang.Thread.run(Unknown Source)
[2009-05-11 08:19:50] [info]

I'm also open to suggestions on tuning my socket performance, if there's 
anything that can be done with it, though I'm aware it's likely the OS, 
and not tomcat that is the issue.

Dave



---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org


Mime
View raw message