tomcat-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Antonio Fiol BonnĂ­n <fiol.bon...@terra.es>
Subject Re: Tomcat stops handling HTTP connections, socket is SYN_RECV
Date Wed, 07 Jan 2004 20:00:30 GMT
I have not read through all the thread, so maybe I am repeating 
something. If so, sorry about being lazy.

You seem to be hitting maxProcessors. If you do, no more threads are 
created, and connections start getting into SYN_RECV state, simply 
because Tomcat is not accept()ing them, exactly as you told it to do.

So, either reduce your concurrency level for the tests, or increase 
maxProcessors.

Maybe there is also sth wrong with the (deprecated) connector code, as 
suggested by someone else. But even with non-deprecated connectors and 
recent Tomcats, you will have a very bad user experience if you hit 
maxProcessors.

If you were using Apache in front of Tomcat (which I know is not your 
case), thigs get even worse. In that case, you must ensure that Apache's 
MaxClients is below Tomcat's maxProcessors, and if you have a firewall 
in the middle, you must also ensure that you are using 
worker.something.socket_timeout=300 or something like that in 
workers.properties, so that your firewall does not mess things up. 
(Assuming mod_jk)

Hope that helps.


Antonio Fiol


Frode E. Moe wrote:

>On Mon, Oct 27, 2003 at 13:29:50 +0100, Remy Maucherat wrote:
>  
>
>>>On Fri, Oct 24, 2003 at 16:01:48 +0200, frode@coretrek.no wrote:
>>>      
>>>
>>>>When everything quiets down, tomcat seems to be unable to service
>>>>any http requests. Any attempt to connect to port 10003 with a web
>>>>browser eventually causes a "No data in response"-style error message.
>>>>At this point it seems the http connector listening on port 10003 has
>>>>gone "zombie" on me: 
>>>>        
>>>>
>>>I've figured out what's going wrong.
>>>The following exception is printed once on standard out when things
>>>fail:
>>>
>>>java.lang.OutOfMemoryError: unable to create new native thread
>>>       at java.lang.Thread.start(Native Method)
>>>       at 
>>>       org.apache.catalina.connector.http.HttpProcessor.threadStart(HttpProcessor.java:1178)
>>>       at 
>>>       org.apache.catalina.connector.http.HttpProcessor.start(HttpProcessor.java:1262)
>>>       at 
>>>       org.apache.catalina.connector.http.HttpConnector.newProcessor(HttpConnector.java:937)
>>>       at 
>>>       org.apache.catalina.connector.http.HttpConnector.createProcessor(HttpConnector.java:868)
>>>       at 
>>>       org.apache.catalina.connector.http.HttpConnector.run(HttpConnector.java:1075)
>>>
>>>I looked into the source for HttpProcessor briefly; perhaps the
>>>"thread.start()" on line 1178 or the threadStart() call on line 1262 
>>>could be surrounded with a try/catch(OutOfMemoryError), and throw a 
>>>LifecycleException?
>>>      
>>>
>>Any reason for using this connector ? It's deprecated.
>>
>>BTW, once there's an OOM exception, the VM is in an unpredictable state, 
>>and must be restarted.
>>    
>>
> 
>If it is of any interest, I have recently started experiencing the same problem 
>again, while doing some profiling work on a webapp. 
>
>I'm now running Tomcat 4.1.29 on Linux 2.4.24 (i386) with Sun Java
>1.4.2_02. I have also changed server.xml to use the newer Coyote
>connectors:
>
>    <Connector className="org.apache.coyote.tomcat4.CoyoteConnector"
>               port="10003" minProcessors="50" maxProcessors="500"
>               enableLookups="true" redirectPort="8443"
>               acceptCount="10" debug="100" connectionTimeout="60000"
>               useURIValidationHack="false" disableUploadTimeout="true" />
>
>When running with the following java options:
> "-Xrunhprof cpu=samples,thread=y;depth=10,cutoff=0,format=a,file=/tmp/log.txt -Xms64m
-Xmsm1024m"
>and "pounding" the webapp with 1000 HTTP requests simulatenously, I got
>the following on System.out:
>
>07.jan.2004 09:27:41 org.apache.tomcat.util.threads.ThreadPool$ControlRunnable run
>SEVERE: Caught exception executing org.apache.tomcat.util.net.TcpWorkerThread@3461d1,
terminating thread
>java.lang.OutOfMemoryError: unable to create new native thread
>        at java.lang.Thread.start(Native Method)
>        at org.apache.tomcat.util.threads.ThreadPool$ControlRunnable.<init>(ThreadPool.java:630)
>        at org.apache.tomcat.util.threads.ThreadPool.openThreads(ThreadPool.java:505)
>        at org.apache.tomcat.util.threads.ThreadPool.findControlRunnable(ThreadPool.java:324)
>        at org.apache.tomcat.util.threads.ThreadPool.runIt(ThreadPool.java:304)
>        at org.apache.tomcat.util.net.TcpWorkerThread.runIt(PoolTcpEndpoint.java:557)
>        at org.apache.tomcat.util.threads.ThreadPool$ControlRunnable.run(ThreadPool.java:666)
>        at java.lang.Thread.run(Thread.java:534)
>
>..after which any attempt to connect to the http port (10003 in this
>case) results in java holding a half-opened connection in SYN_RECV, 
>as shown by "netstat -apeln | grep 10003":
>
>tcp        0      0 0.0.0.0:10003           0.0.0.0:*               LISTEN      1001 
     47392370   27230/java          
>tcp        0      0 127.0.0.1:10003         127.0.0.1:41269         SYN_RECV    1001 
     0          -
>(and several in CLOSE_WAIT state)
>
>Other threads keep running, so the VM _seems_ stable, except for the
>fact that it's no longer possible to connect to the HTTP port.
>
>To me, it seems like it should be possible to recover from this error.
>Any comments?
>
>
>
>---------------------------------------------------------------------
>To unsubscribe, e-mail: tomcat-user-unsubscribe@jakarta.apache.org
>For additional commands, e-mail: tomcat-user-help@jakarta.apache.org
>
>
>  
>


Mime
View raw message