tomcat-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Rainer Jung <rainer.j...@kippdata.de>
Subject Re: TCP Connection Aborted problems
Date Mon, 29 Jan 2007 11:12:37 GMT
I had a look at your log files. It really looks like something is wrong 
on the remote=tomcat side (or in between mod_jk and tomcat). Please do a 
couple of thread dumps for the tomcat JVM immediately after your stress 
test, during the time you cannot send request via mod_jk.

Under Windows you have to run Tomcat from the console in order to be 
able to do a JVM thread dump. You create a thread dump by Typing 
Ctrl-Break. It should go to your catalina.out. It start with a line 
containing 'ull thread'.

Regards,

Rainer

JNeuhoff wrote:
> As a followup from the
> http://www.nabble.com/mod_jk-replacement--tf3050993.html I am still looking
> for a working solution of the err=-53 (TCP Connection aborted) problem.
> 
> To summarize what is happening: 
> 
> We have a Windows 2003 server (with Apache 2.0.59, mod_jk 1.2.20 and Tomcat
> 5.5.17).
> 
> It was subjected by 2 client workstations from the same subnet to a
> stresstest
> involving 2 users. These were rapidly clicking on weblinks within our
> website. Their click-rate was faster on purpose than the Tomcat service
> could handle. The average response time from our servlet
> is about 0.15 seconds, but can go up to 0.5 seconds or more
> depending on server load and how many MySQL database queries
> it has to do.
> 
> After just a minute or two, the server became unresponsive.
> Even though both Apache and Tomcat continued to run and receive
> requests the underlying TCP connections for the responses
> resulted in err=53 or err=54 conditions, hence the web browsing
> clients became unable to receive the responses. Even a re-start
> of the Apache and Tomcat NT services didn't clear up this stalemate
> scenario with the TCP connections.
> 
> Judgeing by the mod_jk.log entries, mod_jk tries to connect to Tomcat, and
> part of it is doing a ping to Tomcat and waiting in vain for a pong. Is
> there a way to configure the connection pool so that it will really clear up
> stale TCP connections before re-using them?
> 
> We using the following configuration:
> 
> httpd.conf:
> 
> ....
> Timeout 300
> KeepAlive On
> MaxKeepAliveRequests 100
> KeepAliveTimeout 15
> # WinNT MPM
> # ThreadsPerChild: constant number of worker threads in the server process
> # MaxRequestsPerChild: maximum  number of requests a server process serves
> <IfModule mpm_winnt.c>
> ThreadsPerChild 250
> MaxRequestsPerChild  0
> </IfModule>
> ....
> ### mod_jk 1.x configuration for connecting to Tomcat 5.5                           
     
>                                                                                     
     
> # Load mod_jk module                                                                
     
> # Update this path to match your modules location                                   
     
> LoadModule    jk_module  modules/mod_jk.so                                          
     
>                                                                                     
     
> Declare the module for <IfModule directive>                                   
           
> #AddModule     mod_jk.c                                                             
     
>                                                                                     
     
> # Where to find workers.properties                                                  
     
> # Update this path to match your conf directory location (put
> workers.properties next to h
> JkWorkersFile "C:/Program Files/Apache Software Foundation/Tomcat
> 5.5/conf/workers.propert
>                                                                                     
     
> # Where to put jk logs                                                              
     
> # Update this path to match your logs directory location (put mod_jk.log
> next to access_lo
> JkLogFile     "C:/Program Files/Apache Software Foundation/Tomcat
> 5.5/logs/mod_jk.log"    
>                                                                                     
     
> # Set the jk log level [debug/error/info]                                           
     
> #JkLogLevel    info                                                                 
     
> JkLogLevel    debug                                                                 
     
>                                                                                     
     
> # Select the log format                                                             
     
> JkLogStampFormat "[%a %b %d %H:%M:%S %Y] "                                          
     
>                                                                                     
     
> # JkOptions indicate to send SSL KEY SIZE,                                          
     
> JkOptions     +ForwardKeySize +ForwardURICompat -ForwardDirectories                 
     
>                                                                                     
     
> # JkRequestLogFormat set the request format                                         
     
> JkRequestLogFormat     "%w %V %T %r %s"                                             
     
>                                                                                     
     
> # Send everything for context /jsp-examples to worker named worker1 (ajp13)         
     
> JkMount  /jsp-examples/* ajp13                                                      
     
>                                                                                     
     
> # Send everything for context /servlets-examples to worker named worker1
> (ajp13)          
> JkMount  /servletsp-examples/* ajp13                                                
     
>                                                                                     
     
> # Send everything for context /manager to worker named worker1 (ajp13)              
     
> JkMount  /manager/* ajp13                                                           
     
>                                                                                     
     
> # Send everything for context /ohpr to worker named worker1 (ajp13)                 
     
> JkMount  /ohpr/* ajp13                                                              
     
>                                                                                     
     
> # Send everything for context /demo-b to worker named worker1 (ajp13)               
     
> JkMount  /demo-b/* ajp13                                                            
     
> ......                                                                              
           
> 
> 
> 
> workers.properties:
> 
> # Define 1 real worker using ajp13
> worker.list=ajp13
> 
> # Set properties for worker1 (ajp13)
> worker.ajp13.type=ajp13
> worker.ajp13.host=localhost
> worker.ajp13.port=8009
> worker.ajp13.connection_pool_timeout=600
> worker.ajp13.connection_pool_minsize=10
> worker.ajp13.connect_timeout=15000
> worker.ajp13.prepost_timeout=10000
> 
> 
> 
> server.xml:
> 
> 
> ....
>     <!-- Define an AJP 1.3 Connector on port 8009 -->
>     <Connector port="8009"
>                enableLookups="false" redirectPort="8443" protocol="AJP/1.3"
> connectionTimeout="600000"/>
> ....
> 
> 
> web.xml for our servlet application:
> 
> ....
>     <session-config>
>       <session-timeout>15</session-timeout>    <!-- 15 minutes -->
>     </session-config>
> ....
> 
> 
> 
> mod_jk.log extract illustrating a blocked TCP connection between Apache and
> Tomcat:
> 
> .....
> 
> [Wed Jan 24 13:02:26 2007] [0284:3868] [debug] jk_uri_worker_map.c (575):
> Attempting to map URI '/ohpr/servlet/Browse' from 5 maps
> [Wed Jan 24 13:02:26 2007] [0284:3868] [debug] jk_uri_worker_map.c (587):
> Attempting to map context URI '/servletsp-examples/*=ajp13' source 'JkMount'
> [Wed Jan 24 13:02:26 2007] [0284:3868] [debug] jk_uri_worker_map.c (587):
> Attempting to map context URI '/jsp-examples/*=ajp13' source 'JkMount'
> [Wed Jan 24 13:02:26 2007] [0284:3868] [debug] jk_uri_worker_map.c (587):
> Attempting to map context URI '/manager/*=ajp13' source 'JkMount'
> [Wed Jan 24 13:02:26 2007] [0284:3868] [debug] jk_uri_worker_map.c (587):
> Attempting to map context URI '/demo-b/*=ajp13' source 'JkMount'
> [Wed Jan 24 13:02:26 2007] [0284:3868] [debug] jk_uri_worker_map.c (587):
> Attempting to map context URI '/ohpr/*=ajp13' source 'JkMount'
> [Wed Jan 24 13:02:26 2007] [0284:3868] [debug] jk_uri_worker_map.c (602):
> Found a wildchar match '/ohpr/*=ajp13'
> [Wed Jan 24 13:02:26 2007] [0284:3868] [debug] mod_jk.c (1999): Into handler
> jakarta-servlet worker=ajp13 r->proxyreq=0
> [Wed Jan 24 13:02:26 2007] [0284:3868] [debug] jk_worker.c (114): found a
> worker ajp13
> [Wed Jan 24 13:02:26 2007] [0284:3868] [debug] jk_worker.c (290): Found
> worker type 'ajp13'
> [Wed Jan 24 13:02:26 2007] [0284:3868] [debug] mod_jk.c (584): Service
> protocol=HTTP/1.1 method=GET host=daohpr1.essex.ac.uk addr=155.245.69.201
> name=daohprw2-a port=80 auth=(null) user=(null) laddr=155.245.69.32
> raddr=155.245.69.201
> [Wed Jan 24 13:02:26 2007] [0284:3868] [debug] jk_ajp_common.c (2315):
> acquired connection pool slot=78
> [Wed Jan 24 13:02:26 2007] [0284:3868] [debug] jk_ajp_common.c (548): ajp
> marshaling done
> [Wed Jan 24 13:02:26 2007] [0284:3868] [debug] jk_ajp_common.c (1771):
> processing ajp13 with 2 retries
> [Wed Jan 24 13:02:26 2007] [0284:3868] [debug] jk_connect.c (335): socket
> TCP_NODELAY set to On
> [Wed Jan 24 13:02:26 2007] [0284:3868] [debug] jk_connect.c (433): trying to
> connect socket 5776 to 127.0.0.1:8009
> [Wed Jan 24 13:02:26 2007] [0284:3868] [debug] jk_connect.c (459): socket
> 5776 connected to 127.0.0.1:8009
> [Wed Jan 24 13:02:26 2007] [0284:3868] [debug] jk_ajp_common.c (847):
> Connected socket 5776 to (127.0.0.1:8009)
> [Wed Jan 24 13:02:26 2007] [0284:3868] [debug] jk_ajp_common.c (893):
> sending to ajp13 pos=4 len=5 max=16
> [Wed Jan 24 13:02:26 2007] [0284:3868] [debug] jk_ajp_common.c (893): 0000   
> 12 34 00 01 0A 00 00 00 00 00 00 00 00 00 00 00  - .4..............
> [Wed Jan 24 13:02:41 2007] [0284:3868] [info]  jk_ajp_common.c (807):
> timeout in reply pong
> [Wed Jan 24 13:02:41 2007] [0284:3868] [info]  jk_ajp_common.c (1259):
> (ajp13) error connecting to the backend server (errno=0)
> [Wed Jan 24 13:02:41 2007] [0284:3868] [info]  jk_ajp_common.c (1916):
> (ajp13) sending request to tomcat failed,  recoverable operation attempt=1
> [Wed Jan 24 13:02:41 2007] [0284:3868] [debug] jk_connect.c (335): socket
> TCP_NODELAY set to On
> [Wed Jan 24 13:02:41 2007] [0284:3868] [debug] jk_connect.c (433): trying to
> connect socket 5776 to 127.0.0.1:8009
> [Wed Jan 24 13:02:41 2007] [0284:3868] [debug] jk_connect.c (459): socket
> 5776 connected to 127.0.0.1:8009
> [Wed Jan 24 13:02:41 2007] [0284:3868] [debug] jk_ajp_common.c (847):
> Connected socket 5776 to (127.0.0.1:8009)
> [Wed Jan 24 13:02:41 2007] [0284:3868] [debug] jk_ajp_common.c (893):
> sending to ajp13 pos=4 len=5 max=16
> [Wed Jan 24 13:02:41 2007] [0284:3868] [debug] jk_ajp_common.c (893): 0000   
> 12 34 00 01 0A 00 00 00 00 00 00 00 00 00 00 00  - .4..............
> [Wed Jan 24 13:02:56 2007] [0284:3868] [info]  jk_ajp_common.c (807):
> timeout in reply pong
> [Wed Jan 24 13:02:56 2007] [0284:3868] [info]  jk_ajp_common.c (1259):
> (ajp13) error connecting to the backend server (errno=0)
> [Wed Jan 24 13:02:56 2007] [0284:3868] [info]  jk_ajp_common.c (1916):
> (ajp13) sending request to tomcat failed,  recoverable operation attempt=2
> [Wed Jan 24 13:02:56 2007] [0284:3868] [error] jk_ajp_common.c (1928):
> (ajp13) Connecting to tomcat failed. Tomcat is probably not started or is
> listening on the wrong port
> [Wed Jan 24 13:02:56 2007] [0284:3868] [debug] jk_ajp_common.c (2258):
> recycling connection pool slot=40 for worker ajp13
> [Wed Jan 24 13:02:56 2007] [0284:3868] [info]  jk_ajp_common.c (807):
> timeout in reply pong
> [Wed Jan 24 13:02:56 2007] [0284:3868] [info]  jk_ajp_common.c (1259):
> (ajp13) error connecting to the backend server (errno=0)
> [Wed Jan 24 13:02:56 2007] [0284:3868] [info]  jk_ajp_common.c (1916):
> (ajp13) sending request to tomcat failed,  recoverable operation attempt=2
> [Wed Jan 24 13:02:56 2007] [0284:3868] [error] jk_ajp_common.c (1928):
> (ajp13) Connecting to tomcat failed. Tomcat is probably not started or is
> listening on the wrong port
> [Wed Jan 24 13:02:56 2007] [0284:3868] [debug] jk_ajp_common.c (2258):
> recycling connection pool slot=40 for worker ajp13

---------------------------------------------------------------------
To start a new topic, e-mail: users@tomcat.apache.org
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org


Mime
View raw message