tomcat-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Peter Bauer <peter.ba...@apus.co.at>
Subject Re: All threads are currently busy
Date Thu, 18 Oct 2007 06:52:22 GMT
Am Mittwoch 17 Oktober 2007 schrieb Johnny Kewl:
> ---------------------------------------------------------------------------
> HARBOR: http://coolharbor.100free.com/index.htm
> Now Tomcat is also a cool application server
> ---------------------------------------------------------------------------
> ----- Original Message -----
> From: "Peter Bauer" <peter.bauer@apus.co.at>
> To: <users@tomcat.apache.org>
> Sent: Wednesday, October 17, 2007 10:52 AM
> Subject: All threads are currently busy
>
> > Hi all,
> >
> > System:
> > Tomcat 4.1.30 on Debian sarge
> > java version "1.5.0_11"
> > Java(TM) 2 Runtime Environment, Standard Edition (build 1.5.0_11-b03)
> > Java HotSpot(TM) Client VM (build 1.5.0_11-b03, mixed mode, sharing)
> >
> > in situations with high load, tomcat sometimes stops responding and is
> > not able to recover from that status. In the catalina.out file, i get the
> > following line:
> >
> > SEVERE: All threads (75) are currently busy, waiting. Increase maxThreads
> > (75) or check the servlet status
> >
> > i searched some time in various forums and mailing list archives and
> > found the following links:
> > http://mail-archives.apache.org/mod_mbox/tomcat-users/200408.mbox/%3c2004
> >Aug01.0922.scs.0012@eskimo.com%3e
> > http://marc.info/?l=tomcat-user&m=113297294417902&w=2
> > http://www.tek-tips.com/viewthread.cfm?qid=862552&page=3
> > http://www.mail-archive.com/tomcat-user@jakarta.apache.org/msg127833.html
> >
> > Basically there seem to be 2 possible reasons for this behaviour
> > 1. A bug (?) in Tomcat or the combination of JDK, glibc, whatever, which
> > causes that tomcat does not
> > recover from situations with too much requests. I could not find any
> > information in the release notes
> > of the Tomcat 4.1.x versions.
> > 2. Some kind of deadlock or resource (database connections) availability
> > problem which
> >   keeps the request handling threads from returning from the doPost/Get
> > method.
> >
> > So i made a Threaddump using jstack (attached file). Most of the threads
> > are in
> >
> > - java.lang.Object.wait(long) @bci=0 (Interpreted frame)
> > - java.lang.Object.wait() @bci=2, line=474 (Interpreted frame)
> > - org.apache.catalina.core.StandardWrapper.allocate() @bci=223, line=704
> > (Interpreted frame)
> >
> > but 19 threads are blocked in the bci 0 of the same static synchronized
> > method called removeActiveRoles
> > which is called from the doPost() method.
> >
> > -
> > at.sonorys.dmc.utilities.ActiveRoleRemoval.removeActiveRoles(at.sonorys.d
> >mc.utilities.DatabaseAccess, java.lang.String, java.lang.String, boolean)
> > @bci=0 (Interpreted frame) -
> > at.sonorys.dmc.CommDicora.CommHandleMessageSession.quitSession() @bci=23
> > (Interpreted frame)
> > - at.sonorys.dmc.CommDicora.CommHandleMessageSession.doHandleSessionMsg()
> > @bci=131 (Interpreted frame)
> > - at.sonorys.dmc.CommDicora.CommHandleMessageSession.evalMessage() @bci=1
> > (Interpreted frame)
> > -
> > at.sonorys.dmc.CommDicora.CommDicoraServlet.doHandleMsg(javax.servlet.htt
> >p.HttpServletResponse, javax.servlet.http.HttpServletRequest) @bci=103
> > (Interpreted frame) -
> > at.sonorys.dmc.CommDicora.CommDicoraServlet.doPost(javax.servlet.http.Htt
> >pServletRequest, javax.servlet.http.HttpServletResponse) @bci=142
> > (Interpreted frame)
> >
> >
> > The servlet implements the SingleThreadModel (not my decision). As you
> > can see from the threaddump, all 19 threads are waiting to
> > get into the same static synchronized method. I dont understand this
> > because there must be one thread IN the method
> > which holds the lock of the method? Is it possible for a thread to leave
> > a synchronized method without returning the
> > lock?
>
> I'm guessing here, never used SingleThreadModel..... but I think that what
> may be happening is the TC for efficiency reasons is creating multiple
> instances of the servlet.... otherwise imagine how slow things would be.

Yes, Tomcat creates up to 20 instances of the servlet on demand.

> So its making (MAYBE) 19 instances of the servlet instance and giving each
> one its own thread.... so its *still* doing what it promised (1 thread per
> servlet)
> BUT... unfortunately for you threads all JOIN again at a STATIC class....
> So if ONE thread gets stuck in ......
>         static synchronized method called removeActiveRoles
> they ALL do.

Thats true, i know about that.

> SingleThreadModel does not fix all threading issues.
> Maybe time to just make the app thread safe and throw SingleThreadModel
> away....

I am not responsible for the initial development, i was just making some 
changes to the existing codebase but if it is inevitable to make a "real" 
servlet out of it...

> From the loooks of things, threads never come out of *removeActiveRoles*
> alive ;)

Thats what i can't understand. Is there a thread still in the method (not the 
case if we can belive the @bci = 0 in the Threaddump) or has a Thread left 
the method in such a way that the lock of the method has not been released? 
So it would not appear in the threaddump but hold the method lock forever. I 
don't know if this is possible.
Even if it was not a SingleThreadModel servlet, every Thread started by tomcat 
to process incoming requests would be stuck waiting for the method lock to be 
released.

thanks,
Peter

> > br,
> > Peter
>
> ---------------------------------------------------------------------------
>-----
>
> > ---------------------------------------------------------------------
> > To start a new topic, e-mail: users@tomcat.apache.org
> > To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
> > For additional commands, e-mail: users-help@tomcat.apache.org
>
> ---------------------------------------------------------------------
> To start a new topic, e-mail: users@tomcat.apache.org
> To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
> For additional commands, e-mail: users-help@tomcat.apache.org



-- 
Peter Bauer
APUS Software G.m.b.H.
A-8074 Raaba, Bahnhofstrasse 1/1
Email: peter.bauer@apus.co.at
Tel: +43 316 401629 24
Fax: +43 316 401629 9

---------------------------------------------------------------------
To start a new topic, e-mail: users@tomcat.apache.org
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org


Mime
View raw message