commons-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Wm.A.Stafford" <staff...@marine.rutgers.edu>
Subject Re: DBCP Locking Up?
Date Tue, 04 Dec 2007 18:55:57 GMT
Torsten,

    We were seeing the same behavior in a lightly loaded application 
when the connection pool was exhausted. I spent several days trying to 
figure out what was happening.  A google search turned up several forum 
and mailing list postings reporting the same problem.  I eventually came 
to the conclusion that the deadlock is some sort of undocumented 
behavior of DBCP that occurs when the pool can no longer fulfill a 
request for a connection.  This was with dbcp-1.2.2 and commons-pool-1.3.  

    I was pretty sure our pool exhaustion was caused by a connection 
leak because we used dbcp to replace a non-pooling DataSource.  So 
connections which would have formerly been garbage collected if not 
closed now were leaked.   We located the leaks by wrapping DBCP with 
class that logged  the state of the pool for every open and close, ex: 
log.info(" active connections=" + pool.getNumActive() + " idle 
connections=" + pool.getNumIdle()) ;

    Eliminating the connection leak eliminated our deadlocks but I never 
figured out what the underlying DBCP problem is. 

    -=bill

Torsten.Romer@teliasonera.com wrote:
> We are using DBCP with iBatis in a JSF/Ajax webapp running in Tomcat with an Oracle database.
>
> The application has quite high load and concrrency, and the database is very loaded.
The application is running fine for some days but then freezes at peak hours.
>
> When I look at a thread dump made while the app hangs, I can see as many threads as we
have connections in the pool (64) in BLOCKED state while calling either borrowObject() or
returnObject(), please see the two examples below. It looks like some kind of deadlock and
the only way out of it seems to be a restart of Tomcat.
>
> First I thought this is normal with the WHEN_EXHAUSTED_LOCK action, but since we have
maxWait set to 1000 I'd not expect this kind of blocking to happen.
>
> I tried to reproduce the behavior with a small test app, but here the threads go in TIMED_WAIT
state when the pool is exhausted and become RUNNABLE again as soon as some connection became
available again.
>
> I really don't understand what is happening and why the pool gets locked up that way.
>
> Any hint/suggestion welcome!
>
> Torsten
>
> "TP-Processor483" daemon prio=10 tid=0x01fe5000 nid=0x11f7 waiting for monitor entry
[0x8d87e000..0x8d881788]
>    java.lang.Thread.State: BLOCKED (on object monitor)
> 	at org.apache.commons.pool.impl.GenericObjectPool.returnObject(GenericObjectPool.java:916)
> 	- waiting to lock <0x9ddf7460> (a org.apache.commons.pool.impl.GenericObjectPool)
> 	at org.apache.commons.dbcp.PoolableConnection.close(PoolableConnection.java:87)
> 	- locked <0xf83f9eb8> (a org.apache.commons.dbcp.PoolableConnection)
> 	at org.apache.commons.dbcp.PoolingDataSource$PoolGuardConnectionWrapper.close(PoolingDataSource.java:181)
> 	at com.ibatis.sqlmap.engine.transaction.jdbc.JdbcTransaction.close(JdbcTransaction.java:81)
> 	at com.ibatis.sqlmap.engine.transaction.TransactionManager.end(TransactionManager.java:110)
>
> "TP-Processor382" daemon prio=10 tid=0x026f6000 nid=0x25a waiting for monitor entry [0x7de7f000..0x7de81808]
>    java.lang.Thread.State: BLOCKED (on object monitor)
> 	at org.apache.commons.pool.impl.GenericObjectPool.borrowObject(GenericObjectPool.java:781)
> 	- waiting to lock <0x9ddf7460> (a org.apache.commons.pool.impl.GenericObjectPool)
> 	at org.apache.commons.dbcp.PoolingDataSource.getConnection(PoolingDataSource.java:96)
> 	at org.apache.commons.dbcp.BasicDataSource.getConnection(BasicDataSource.java:880)
> 	at com.ibatis.sqlmap.engine.transaction.jdbc.JdbcTransaction.init(JdbcTransaction.java:48)
> 	at com.ibatis.sqlmap.engine.transaction.jdbc.JdbcTransaction.getConnection(JdbcTransaction.java:89)
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: user-unsubscribe@commons.apache.org
> For additional commands, e-mail: user-help@commons.apache.org
>
>   

-- 

Wm.A.Stafford

Principal Engineer

The Institute of Marine and Coastal Sciences
Rutgers University
732-932-6555 x367
stafford@marine.rutgers.edu

---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@commons.apache.org
For additional commands, e-mail: user-help@commons.apache.org


Mime
View raw message