tomcat-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Daniel Mikusa <dmik...@vmware.com>
Subject Re: Analyzing Connection Pool Errors/Leaks
Date Mon, 01 Apr 2013 21:11:50 GMT
On Apr 1, 2013, at 4:18 PM, David Landis wrote:

> Thanks for the response, see my comments inline below.
> 
> 
> On Mon, Apr 1, 2013 at 3:49 PM, Daniel Mikusa <dmikusa@vmware.com> wrote:
> 
>> On Apr 1, 2013, at 3:31 PM, David Landis wrote:
>> 
>>> Hi guys,
>>> 
>>> When running a performance test on my system it starts fine, but after a
>>> while I start getting errors in my application log such as (see the
>> bottom
>>> for full stack trace):
>>> 
>>> 2013-03-29 16:38:54,778 ERROR
>>> [org.hibernate.engine.jdbc.spi.SqlExceptionHelper] -
>>> [SimpleAsyncTaskExecutor-12842] - [SimpleAsyncTaskExecutor-12842]
>> Timeout:
>>> Pool empty. Unable to fetch a connection in 30 seconds, none
>>> available[size:80; busy:0; idle:0; lastwait:30000].
>> 
>> This means you have no connections in your pool and it's unable to create
>> a new connection to your database.
>> 
> 
> 
> OK, I'll have to investigate the DB setting more thoroughly. The maximum
> sessions and processes in Oracle are higher than we were using for the test
> though (several hundred).
> 
> 
> 
>> 
>>> 
>>> Questions:
>>> 
>>> 1.) I'm a little confused about what it means if no connections are
>>> available and yet none are "busy" nor "idle".  What are the other
>> available
>>> states?
>> 
>> The pool is empty.  Further more the error above means that it can't
>> create a new connection either.  Maybe your network failed? or the DB
>> kicked off all your application's connections?
>> 
> 
> 
> Actually Oracle was showing 70+ inactive sessions for my app even though
> the connection pool was showing empty.

Possible you are hitting a bug.

You might also want to try an upgrade of Tomcat.  You're a couple versions back at 7.0.32.
 You can see what was fixed by searching for "jdbc-pool" in the ChangeLog.

  https://tomcat.apache.org/tomcat-7.0-doc/changelog.html

Dan


> 
> 
>> Were you ever able to get a connection to the DB? If you restart Tomcat,
>> can you get connections to the DB again?
>> 
> 
> 
> Yes, restarting Tomcat results in a fresh pool of DB connections and the
> 70+ inactive sessions on the DB side are gone and replaced by 10 which is
> the initial size of the pool.
> 
> 
> 
>> 
>> Also, are there any limits on your DB user's account that might cause
>> problems with your performance tests?
>> 
> 
> 
> Not that I know of, but I'll look further. I was expecting problems with
> the perf test eventually b/c it was set to simulate a couple hundred users
> and I only maxActive set to 80. That is fine. I'm more concerned with why
> the connection pool didn't eventually recover.
> 
> 
> 
>> 
>>> 
>>> 2.) My other point of confusion is that assuming there is a connection
>> leak
>>> in the application, shouldn't  setting removeAbandoned=true cause the DB
>>> connections to eventually be recovered?
>> 
>> Yes.
>> 
>>> What I am seeing is that even after
>>> a couple days of no application usage now I'm still getting that exact
>>> error seen above when trying to use the application.
>> 
>> See above, something bad is happening in your environment, worse than your
>> application holding onto a connection.  You're going to need to
>> troubleshoot further and see why you can't make connections to the DB.
>> 
>> Dan
>> 
>> 
>> 
>>> 
>>> Thanks in advance.
>>> 
>>> ----------------------------------------
>>> 
>>> environment:
>>> Tomcat 7.0.32
>>> RedHat Linux 64 bit
>>> Java 7
>>> 
>>> context.xml:
>>> 
>>> <Resource name="fdatasource"
>>> auth="Container"
>>> type="javax.sql.DataSource"
>>> factory="org.apache.tomcat.jdbc.pool.DataSourceFactory"
>>> testWhileIdle="true"
>>> testOnBorrow="true"
>>> testOnReturn="true"
>>> validationQuery="select 1 from dual"
>>> validationInterval="30000"
>>> timeBetweenEvictionRunsMillis="5000"
>>> maxActive="80"
>>> minIdle="10"
>>> maxWait="30000"
>>> initialSize="10"
>>> removeAbandonedTimeout="1200"
>>> removeAbandoned="true"
>>> logAbandoned="true"
>>> minEvictableIdleTimeMillis="60000"
>>> jmxEnabled="false"
>>> jdbcInterceptors="StatementFinalizer"
>>> username="${db.user}"
>>> password="${db.pw}"
>>> driverClassName="oracle.jdbc.OracleDriver"
>>> url="${db.url}" />
>>> 
>>> example stacktrace:
>>> 
>>> Mar 29, 2013 5:13:34 PM org.apache.catalina.core.StandardWrapperValve
>> invoke
>>> SEVERE: Servlet.service() for servlet [dispatcher] in context with path
>>> [/app] threw exception [Request processing failed; nested exception is
>>> org.springframework.transaction.CannotCreateTransactionException: Could
>> not
>>> open Hibernate Session for transaction; nested exception is
>>> org.hibernate.exception.GenericJDBCException: Could not open connection]
>>> with root cause
>>> org.apache.tomcat.jdbc.pool.PoolExhaustedException:
>>> [http-apr-8443-exec-158] Timeout: Pool empty. Unable to fetch a
>> connection
>>> in 30 seconds, none available[size:84; busy:0; idle:0; lastwait:30000].
>>> at
>>> 
>> org.apache.tomcat.jdbc.pool.ConnectionPool.borrowConnection(ConnectionPool.java:675)
>>> at
>>> 
>> org.apache.tomcat.jdbc.pool.ConnectionPool.getConnection(ConnectionPool.java:188)
>>> at
>>> 
>> org.apache.tomcat.jdbc.pool.DataSourceProxy.getConnection(DataSourceProxy.java:128)
>>> at
>>> 
>> org.hibernate.service.jdbc.connections.internal.DatasourceConnectionProviderImpl.getConnection(DatasourceConnectionProviderImpl.java:141)
>>> at
>>> 
>> org.hibernate.internal.AbstractSessionImpl$NonContextualJdbcConnectionAccess.obtainConnection(AbstractSessionImpl.java:292)
>>> at
>>> 
>> org.hibernate.engine.jdbc.internal.LogicalConnectionImpl.obtainConnection(LogicalConnectionImpl.java:297)
>>> at
>>> 
>> org.hibernate.engine.jdbc.internal.LogicalConnectionImpl.getConnection(LogicalConnectionImpl.java:169)
>>> at
>>> 
>> org.hibernate.engine.transaction.internal.jdbc.JdbcTransaction.doBegin(JdbcTransaction.java:67)
>>> at
>>> 
>> org.hibernate.engine.transaction.spi.AbstractTransactionImpl.begin(AbstractTransactionImpl.java:160)
>>> at
>>> 
>> org.hibernate.internal.SessionImpl.beginTransaction(SessionImpl.java:1395)
>>> at
>>> 
>> org.springframework.orm.hibernate4.HibernateTransactionManager.doBegin(HibernateTransactionManager.java:387)
>> 
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
>> For additional commands, e-mail: users-help@tomcat.apache.org
>> 
>> 

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org


Mime
View raw message