tomcat-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Ajit Narayanan" <aj...@techmahindra.com>
Subject RE: FW: tomcat hangs with error msg-"All Threads are Busy....."
Date Thu, 18 May 2006 11:14:28 GMT

You are right! 

The code does implement a mutex() part. But the problem lasted for 7 days
untill the server was restarted manually.

There were about 15 threads which were found to be in locked state.And for
all these days tomcat was not accepting any requests.

Do you think Dev that this could have happened under the conditions that you
have mentioned .


And the following message was found in the catalina.out log file:

Dec 7, 2004 3:44:37 PM 
org.apache.tomcat.util.threads.ThreadPool logFull
                SEVERE: All threads are busy, waiting. Please increase
maxThreads or check the servlet status75 75  








-----Original Message-----
From: David Delbecq [mailto:delbd@oma.be] 
Sent: Thursday, May 18, 2006 2:35 PM
To: Tomcat Users List
Subject: Re: FW: tomcat hangs with error msg-"All Threads are Busy....."

ok, i bet then you problem is in this :D:

au.com.AdmTech.incontrol.secureproxy.SecureProxy.doGet(SecureProxy.java:298)
	- locked <0xf227ea88> (a
au.com.alcatel.incontrol.secureproxy.SecureProxy)
	at
au.com.AdMtech.incontrol.secureproxy.SecureProxy.doPost(SecureProxy.java:435
)

it uses a lock to a ressource or there is a synchronized method in the way.
As such, until one process has finished, others can't start working. 
This is bad :)
As a consequence, if one of clients is not responding but not sending data
to the wire, it locks everything.
In normal course of operation, you end up with a connection closed exception
in socketinputstream.read() but it takes time to get them. and the delay is
platform dependant. if client abruptly close, TCP/IP protocol has no
information  about this except for a very long delay.

Maybe some protocol level tunings like keep alives might help :/

Anyway, this has nothing to do with GC :)

Ajit Narayanan wrote:
> Thanx for ur interest Dav.I am fairly new to java and tomcat 
> programming so I am bit low on debugging these.
>
> Yes,I do have some of the data from the dumps, I don't know whether 
> these will be of any help or not but this what I have.This happened on 
> live about
> 7 months ago and it has been reopened on cux demand ,a workaround has 
> been patched where in the socket gets realesed after some 90- secs.The 
> following dump has been taken before this workaround:
>
> "Thread-540" daemon prio=5 tid=0x0048b9e0 nid=0x33a runnable 
> [ee47f000..ee4819a8]
> 	at java.net.SocketInputStream.socketRead0(Native Method)
> 	at java.net.SocketInputStream.read(Unknown Source)
> 	at
> org.apache.coyote.http11.InternalInputBuffer.fill(InternalInputBuffer.
> java:7
> 77)
> 	at
> org.apache.coyote.http11.InternalInputBuffer$InputStreamInputBuffer.do
> Read(I
> nternalInputBuffer.java:807)
> 	at
> org.apache.coyote.http11.filters.IdentityInputFilter.doRead(IdentityIn
> putFil
> ter.java:158)
> 	at
> org.apache.coyote.http11.InternalInputBuffer.doRead(InternalInputBuffe
> r.java
> :742)
> 	at org.apache.coyote.Request.doRead(Request.java:431)
> 	at
> org.apache.coyote.tomcat4.CoyoteInputStream.readBytes(CoyoteInputStrea
> m.java
> :199)
> 	at
>
org.apache.coyote.tomcat4.CoyoteInputStream.read(CoyoteInputStream.java:156)
> 	at
> org.apache.coyote.tomcat4.CoyoteRequest.readPostBody(CoyoteRequest.jav
> a:1983
> )
> 	at
> org.apache.coyote.tomcat4.CoyoteRequest.parseRequestParameters(CoyoteR
> equest
> .java:1965)
> 	at
>
org.apache.coyote.tomcat4.CoyoteRequest.getParameter(CoyoteRequest.java:911)
> 	at
> org.apache.coyote.tomcat4.CoyoteRequestFacade.getParameter(CoyoteReque
> stFaca
> de.java:178)
> 	at
>
au.com.AdmTech.incontrol.secureproxy.SecureProxy.doGet(SecureProxy.java:298)
> 	- locked <0xf227ea88> (a
> au.com.alcatel.incontrol.secureproxy.SecureProxy)
> 	at
> au.com.AdMtech.incontrol.secureproxy.SecureProxy.doPost(SecureProxy.ja
> va:435
> )
> 	at javax.servlet.http.HttpServlet.service(HttpServlet.java:760)
> 	at javax.servlet.http.HttpServlet.service(HttpServlet.java:853)
>
> And there are 71 threads waiting to lock <0xf227ea88>:
>
> "- waiting to lock <0xf227ea88>
>
> Apart from what you suggested can this be a problem of compatibility 
> between tomcat-JDK and the POSIX lib of solaris?
> Or can it be that the gc is not able to execute as generally the load 
> on this proxy is very high leading to such a problem.
>
> Thanks and regards,
> Ajit.
>   
>
> -----Original Message-----
> From: David Delbecq [mailto:delbd@oma.be]
> Sent: Thursday, May 18, 2006 1:32 PM
> To: Tomcat Users List
> Subject: Re: FW: tomcat hangs with error msg-"All Threads are Busy....."
>
> This most probably means there is deadlock somewhere in your 
> webapplication, leading to HTTP threads  not being released because 
> they are stuck waiting for something before sending response to a user
that dropped hours agos.
>
> To get clues on where your problem is when this happens, send signal 
> SIGQUIT to tomcat process, it will force the sun JVM to dump all threads
stacks.
>
> Regards,
> David Delbecq
> Ajit Narayanan wrote:
>   
>>  
>> hi,
>>  
>> My tomcat 4.1.27 is running on sun Solaris 8 box but ocassionally it 
>> hangs out giving the error " All threads are busy, waiting. Please 
>> increase maxThreads or check the servlet status75".  I have tried 
>> increasing the no of threads but its not working I guess.  The 
>> application code looks clean but can there be any compatibility 
>> issues (of tomcat and JDK) with the solaris POSIX thread libraries?? 
>> If there is  any problem then what can be done to resolve this 
>> problem. One more thing restarting the tomcat manually resolves the
problem.
>>  
>> regards,
>> Ajit.
>>
>>
>> =====================================================================
>> = ======================================================
>>
>> Tech Mahindra, formerly Mahindra-British Telecom.
>>  
>>
>> Disclaimer:
>>
>>
>> The contents of this E-mail (including the contents of the 
>> enclosure(s) or
>>     
> attachment(s) if any) are privileged and confidential material of Tech 
> Mahindra and should not be disclosed to, used by or copied in any 
> manner by anyone other than the intended addressee(s). In case you are 
> not the desired addressee, you should delete this message and/or re-direct
it to the sender.
> The views expressed in this E-mail message (including the enclosure(s) 
> or
> attachment(s) if any) are those of the individual sender, except where 
> the sender expressly, and with authority, states them to be the views 
> of Tech Mahindra.
>   
>> This e-mail message including attachment/(s), if any, is believed to 
>> be
>>     
> free of any virus. However, it is the responsibility of the recipient 
> to ensure that it is virus free and Tech Mahindra is not responsible 
> for any loss or damage arising in any way from its use.
>   
>> =====================================================================
>> = ======================================================
>>   
>>     
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
> For additional commands, e-mail: users-help@tomcat.apache.org
>
>
> ======================================================================
> ======================================================
>
> Tech Mahindra, formerly Mahindra-British Telecom.
>  
>
> Disclaimer:
>
>
> The contents of this E-mail (including the contents of the enclosure(s) or
attachment(s) if any) are privileged and confidential material of Tech
Mahindra and should not be disclosed to, used by or copied in any manner by
anyone other than the intended addressee(s). In case you are not the desired
addressee, you should delete this message and/or re-direct it to the sender.
The views expressed in this E-mail message (including the enclosure(s) or
attachment(s) if any) are those of the individual sender, except where the
sender expressly, and with authority, states them to be the views of Tech
Mahindra.
>
>
> This e-mail message including attachment/(s), if any, is believed to be
free of any virus. However, it is the responsibility of the recipient to
ensure that it is virus free and Tech Mahindra is not responsible for any
loss or damage arising in any way from its use.
>
> ======================================================================
> ======================================================
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
> For additional commands, e-mail: users-help@tomcat.apache.org
>
>   


---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org


============================================================================================================================

Tech Mahindra, formerly Mahindra-British Telecom.
 

Disclaimer:


The contents of this E-mail (including the contents of the enclosure(s) or attachment(s) if
any) are privileged and confidential material of Tech Mahindra and should not be disclosed
to, used by or copied in any manner by anyone other than the intended addressee(s). In case
you are not the desired addressee, you should delete this message and/or re-direct it to the
sender. The views expressed in this E-mail message (including the enclosure(s) or attachment(s)
if any) are those of the individual sender, except where the sender expressly, and with authority,
states them to be the views of Tech Mahindra.


This e-mail message including attachment/(s), if any, is believed to be free of any virus.
However, it is the responsibility of the recipient to ensure that it is virus free and Tech
Mahindra is not responsible for any loss or damage arising in any way from its use.

============================================================================================================================

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org


Mime
View raw message