commons-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From bugzi...@apache.org
Subject DO NOT REPLY [Bug 16581] New: - Deadlock in AbandonedObjectPool
Date Thu, 30 Jan 2003 06:51:00 GMT
DO NOT REPLY TO THIS EMAIL, BUT PLEASE POST YOUR BUG 
RELATED COMMENTS THROUGH THE WEB INTERFACE AVAILABLE AT
<http://nagoya.apache.org/bugzilla/show_bug.cgi?id=16581>.
ANY REPLY MADE TO THIS MESSAGE WILL NOT BE COLLECTED AND 
INSERTED IN THE BUG DATABASE.

http://nagoya.apache.org/bugzilla/show_bug.cgi?id=16581

Deadlock in AbandonedObjectPool

           Summary: Deadlock in AbandonedObjectPool
           Product: Commons
           Version: 1.0 Final
          Platform: Sun
               URL: http://www.newedgenet.com/
        OS/Version: Solaris
            Status: NEW
          Severity: Major
          Priority: Other
         Component: Dbcp
        AssignedTo: commons-dev@jakarta.apache.org
        ReportedBy: gturner@unzane.com


I have a 3 year old web application that at one time had a custom connection 
pool, then upgraded to Tomcat 4.0.x using the Tyrex DataSource, and recently 
upgraded to Tomcat 4.1.x using DBCP.  After a few hours of operation all 
threads hang on AbandonedObjectPool.borrowObject and returnObject.  The 
database is Oracle 8.1.6 using thin driver and it is firewalled.  Solaris 8 on 
sparc, 64-bit JDK 1.4.1_01.

I believe what is happening is that the firewall is closing connections after 
an hour of inactivity.  When a borrowObject has a lock on the pool and is 
restoring the state of a Connection, setAutoCommit(true) blocks indefinately on 
one of these closed sockets.  Meanwhile the other threads are trying to 
borrowObject too, waiting for the pool to unlock, and some threads are even 
trying to returnObject.  So the problem is introduced when the webapp gets a 
lot of attention (but getting nowhere near maxActive), then slows down a little 
so that some open sockets to the database get automatically closed by the 
firewall, and finally the app picks up a little so that a stale socket gets 
reused.

I'm attaching a huge thread dump, that contains the following threads:

  There are a total 100 maxConnection threads

    * 1 of which is hanging on setAutoCommit and has the lock on
      AbandonedObjectPool.borrowObject (Thread-42)

    * 5 of which are trying to returnObject (Thread-32, 18, 15, 10, and 5)

    * 95 other threads all blocked on borrowObject

  There are a few more idle connector threads because the server.xml is
  partitioned for another web application which isn't active.

Earlier today I futilely attempted to set maxIdle to 0 so that all connections 
were closed, it still crashed and I found that returnObject tests for "_maxIdle 
> 0", so perhaps I'll test it with maxIdle set to 1.  That should provide a lot 
longer uptime, but still not 100% safe.

I also decompiled the Oracle driver and confirmed that it makes no attempt to 
setSoTimeout (nor construct with a timeout) and looking at 
java.net.PlainSocketImpl, the default is indeed 0 (blocking).

I believe a good workaround would be to move 
PoolableConnectionFactory.activateObject calling setAutoCommit out of the way 
of path the locked pool.

Soon I'll try sub-classing BasicDataSourceFactory and try enabling the idle 
eviction features of GenericObjectPool.

---------------------------------------------------------------------
To unsubscribe, e-mail: commons-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: commons-dev-help@jakarta.apache.org


Mime
View raw message