commons-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jason Lea <ja...@kumachan.net.nz>
Subject Re: [DBCP] Please let us start to improve the connection pool
Date Tue, 25 Mar 2003 02:06:06 GMT
Craig R. McClanahan wrote:
> 
> On Mon, 24 Mar 2003, Danny Angus wrote:
> 
> 
>>Date: Mon, 24 Mar 2003 22:28:29 -0000
>>From: Danny Angus <danny@apache.org>
>>Reply-To: Jakarta Commons Developers List <commons-dev@jakarta.apache.org>
>>To: Jakarta Commons Developers List <commons-dev@jakarta.apache.org>
>>Subject: RE: [DBCP] Please let us start to improve the connection pool
>>
>>Craig,
>>
>>
>>>IMHO, any application that depends on the connection pool for recovering
>>>abandoned connections (whether or not it recycles them) is broken.  Far
>>>better is to focus your energy on avoiding all the cases where you grab a
>>>connection from the pool but fail to return it for some reason.
>>>
>>>One simple way to do this is to encapsulate your JDBC-using code in a
>>>try/catch/finally block, something like this:
>>
>>I haven't used DBCP for anthing yet, though we're proposing to use it
>>for James in place of our homespun pool.
>>
>>However from what I understand of this discussion there are two things
>>going on here;
>>
>>Thing one, I agree with you, code failing to return connections to the
>>pool should lead to failure, and the sooner this can be done the better
>>for identifying such broken code.
>>
>>Thing two, DBCP will apparently make a value judgement about an assigned
>>connection, and is capable of recycling it with no notification to the
>>code which has checked it out.
>>
> 
> 
> will --> can be configured to
> 
> 
>>In my opinion thing two is wrong or incomplete as it creates a situation
>>where potential failure is built in, difficult to reproduce and
>>difficult identify the cause of.
>>
> 
> 
> There is a lot of flexibility in how you configure DBCP -- most of it
> inherited from the commons-pool capabilities that lie underneath.  In all
> cases, the use of these features is configurable.
> 
> * Test On Borrow -- The pool can perform some tests on an object instance
>   that the pool is about to return to the application that asked for it.
> 
> * Test On Return -- The pool can perform the same sorts of tests, but this
>   time when the application returns the instance to the pool.
> 
> * Test While Idle -- The pool can perform checks on connections that are
>   in the pool (not currently allocated to an application) to make sure
>   that it is still valid.
> 
> DBCP (in particular, the BasicDataSource class found there) implements
> flags to enable each of these behaviors individually, along with a
> validation query that is used to perform the test.  In addition, there are
> tuning parameters to control things like how often the "test when idle"
> checks are performed, and how many connections are tested each time.
> 
> 
>>In the case of JDBC connection pooling may be reasonable to want to keep
>>a connection even when it is idle, because connections can aquire state
>>which is expensive to reproduce. Is it not, then, unresonable to allow
>>the pool to silently and forcably recycle apparently idle but valuable
>>connections?
>>
> 
> 
> Keeping physical JDBC connections open between uses by the app, and
> allowing those connections to be shared, is the whole point of DBCP :-).
> 
> But I think we might be using the term "recycle" differently.
> 
> By "recycle", do you mean "if a connection has been setting in the pool
> for a long time and is not allocated to an application, so we can close it
> now"?  If so, that behavior is already configurable -- just set the
> maxIdle property to zero and no idle connection will ever get harvested.
> There's no application to notify in this case, because the instance being
> harvested wasn't allocated to an application at the time.
> 
> Or, by "recycle", do you mean "if a connection has been allocated to an
> application but not returned for a long time, the pool is allowed to grab
> it back again -- but you want to notify the application first".  The "grab
> it back" behavior is already configurable (and not enabled by default) --
> and it's this functionality that I object to having at all.  If there was
> no "grab it back" we wouldn't have to worry about notifying anyone that it
> was about to happen :-).
> 
> 
>>My solution would either be to make it possible to turn off the forcable
>>recycling of connections, or to make the pool capable of notifying code
>>that its connection has been recycled.
>>
> 
> 
> You can already turn off the forcible recycling -- in fact, you explicitly
> have to turn it *on* (i.e., for BasicDataSource, you have to explicitly
> set the "removeAbandoned" property and related values to enable it).
> 
> But, I contend that no one should ever do that :-).
> 
> 
>>Is that reasonable?
> 
> 
> If we're talking about the second use of "recycled" above, IMHO, I think
> adding support for recovering abandoned connections at all was a mistake.
> Doing anything to make it work "better" (knowing all the while that it
> cannot be made "perfect") simply perpetuates the mistake.  I'd much rather
> see this whole area of functionality deprecated, rather than continuing to
> mislead people into believing that its OK to depend on something that
> cannot ever work reliably 100% of the time.
> 
> But that's just my opinion.

I think Craig is correct.

If code is not closing connections (and thereby not returning them to 
the pool), it is not the responsibility of the pool to try and recover 
them.  Applications would suffer the same problem if they didn't use 
pooling.  Pooling is there to try and make applications faster and more 
efficient, not to cover up bugs.

If our code is not closing connections or other resources we want to 
find out about this as soon as possible during development.  If 
'abandoned' connections are being recovered by the pool, that obscures 
the fact.  It could lead to the problem causing intermittent connection 
problems in production, exactly where you don't want it.

DBCP (& Pool) should help us see any database connection problems (which 
it is presently hiding) such as problems creating a new db connection, 
evicting an idle connection or a failed validation.

I think effort should be put into this area by adding commons-logging to 
  DBCP and Pool.  In particular, DelegateConnection.passivate() and 
DelegateStatement.passivate() would be good candidates as they close 
open Statements and ResultSets that the application should have closed.

To help locate code which has failed to close a connection, it would be 
nice to have a list of 'active' connections and the length of time they 
have been active.  As I funnel all of my connection requests through a 
static method in my code I have a central place where I can log my 
connection requests.  I will probably add some code to record the 
Connection.hashCode() and the time to a hashmap so I can generate a 
report on the number/time connections are left open.  I will also use a 
static method to close my connections so I can remove from the hashmap 
and log the close().  (I wonder if DBCP/Pool should do such as thing? 
It guess it is really only useful for debugging or possibly displaying 
statistics on pool usage eg average time object is used)


Jason Lea

> 
> 
>>d.
>>
> 
> 
> Craig
> 
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: commons-dev-unsubscribe@jakarta.apache.org
> For additional commands, e-mail: commons-dev-help@jakarta.apache.org
> 
> 


-- 
Jason Lea
Email:     jason@kumachan.net.nz
Phone/Fax: +64  3 381 2907
Mobile:    +64 21 040 2708
Address:   9a Tabart Street, Christchurch, New Zealand


---------------------------------------------------------------------
To unsubscribe, e-mail: commons-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: commons-dev-help@jakarta.apache.org


Mime
View raw message