commons-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Shawn Heisey <apa...@elyograg.org>
Subject Re: [DBCP] Connection pool not behaving as I expect
Date Fri, 02 Mar 2018 05:50:38 GMT
On 3/1/2018 8:48 PM, Matt Sicker wrote:
> Take a look inside commons-pool for the instrumentation (e.g., JMX). You
> can also track usage on borrow and other leaks.
>
> Also, Tomcat uses DBCP as it is.

I know I can get those numbers from the object pool, but at the point 
where I needed them, I don't actually know *which* DataSource is being 
used.  It could be one of two.  So I don't know which object pool to 
look at.

Interim solution before I switch to BasicDataSource:  Log info from BOTH 
pools.  I did this, and here's some info that gets logged on the dev 
server where I installed the change:

WARN  - 2018-03-01 21:59:00.313;   371; c; CP: main-active=0, 
main-idle=1, master-active=0, master-idle=1
WARN  - 2018-03-01 21:59:00.314;   371; c; CP: main-active=0, 
main-idle=1, master-active=0, master-idle=1
WARN  - 2018-03-01 21:59:00.315;   371; c; CP: main-active=0, 
main-idle=1, master-active=0, master-idle=1
WARN  - 2018-03-01 21:59:00.316;   371; c; CP: main-active=0, 
main-idle=1, master-active=0, master-idle=1
WARN  - 2018-03-01 21:59:00.318;   371; c; CP: main-active=0, 
main-idle=1, master-active=0, master-idle=1
WARN  - 2018-03-01 21:59:00.319;   371; c; CP: main-active=0, 
main-idle=1, master-active=0, master-idle=1
WARN  - 2018-03-01 21:59:00.320;   371; c; CP: main-active=0, 
main-idle=1, master-active=0, master-idle=1
WARN  - 2018-03-01 21:59:00.321;   371; c; CP: main-active=0, 
main-idle=1, master-active=0, master-idle=1
WARN  - 2018-03-01 21:59:00.322;   371; c; CP: main-active=0, 
main-idle=1, master-active=0, master-idle=1
WARN  - 2018-03-01 21:59:00.323;   371; c; CP: main-active=0, 
main-idle=1, master-active=0, master-idle=1
WARN  - 2018-03-01 21:59:00.324;   371; c; CP: main-active=0, 
main-idle=1, master-active=0, master-idle=1
WARN  - 2018-03-01 21:59:00.325;   371; c; CP: main-active=0, 
main-idle=1, master-active=0, master-idle=1

This logging happens just before a connection is obtained from the 
pool.  The info logged is the same -- zero active, one or more idle.  
Except during program startup, when there are some lines where both are 
zero.  And once an hour, there is a background thread that does a query 
that takes nearly a minute to run, and while that's happening, I do see 
main-active=1.

I did just now think of one possibility that might explain why this 
program misbehaves when the problem happens, even if DBCP is working 
right.  Based on the logging I've enabled, and the fact that nobody has 
said "oh, we see that all the time, and the problem is probably X" ... I 
think DBCP probably is working right.

====
Basic info about my program: All reads are done via the "main" pool, 
which connects to a slave, unless connections there are not working, 
then reads switch to the "master" pool for the next 1000 connections.  
All writes are done via the "master" pool.

When the master server reaches its connection limit, a completely 
separate process that adds information to the monster table in the DB 
cannot work.  It's not Java-based, and doesn't have connection pooling 
available.

When there are no new docs in the monster table, my program doesn't have 
anything to do, so it doesn't need to make any changes to its control 
table -- it's not going to be using the master pool.  That probably 
results in the idle connection to the master server being evicted five 
minutes after the additions to the database stop.  Then because there's 
now a connection available on the server, the other system can suddenly 
add some documents.  So my program notices the new docs via its main 
pool, processes them, and then it would need to update its control table 
on the master server.  There's no idle connection because it got 
evicted, so it tries to make a new one, and we get an explosion.
====

I still do have the separate problem of why our app servers explode with 
"Too many connections" exceptions, when the DB pools there have dozens 
of active connections.  I didn't write that code.  Maybe their DB access 
(which is primarily through hibernate, a library I don't know much 
about) is not properly releasing connections back to the pool, so the 
pool does not think they're actually idle.  I will need to see what 
logging they can add.

Thanks,
Shawn


---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@commons.apache.org
For additional commands, e-mail: user-help@commons.apache.org


Mime
View raw message