commons-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Shawn Heisey <apa...@elyograg.org>
Subject Re: [DBCP] Apparent deadlock related to getConnection
Date Sun, 12 Apr 2015 17:31:54 GMT
On 4/12/2015 9:16 AM, Phil Steitz wrote:
> Can you post your full pool config and info on versions of DBCP and
> POOL?
> 
> Do you call isClosed() before closing a connection (not recommended
> - that is really the issue in the ticket you commented)?
> 
> To make sure there is not an actual deadlock somewhere, you should
> examine a full thread dump.  The thread you posted in the ticket is
> waiting on the pool, not holding any DBCP or POOL related locks; but
> it does hold this application lock:
> at com.REDACTED.idxbuild.solr.Chain.doReinsert(Chain.java:1332)
>         - locked <0x00000000db0011c8> (a java.lang.Object)
> Make sure no other threads in process of doing things that will
> return connections to the pool aren't waiting on this lock.
> 
> Given that you are using DBCP 2.1, if you have access to a JMX
> console it would also be good to look at what is happening in the
> pool when your application hangs.  In particular, numActive, numIdle
> and listAllObjects would be good to look at.  Looks like you are
> using PoolingDataSource directly, so you need to look at the
> underlying GenericObject's JMX properties.

My code does not contain the string "isClosed" at all.

I thought I had saved the full thread dump, but the stdout logfile was
wiped when I restarted the program, and I didn't think to save a copy.
I will be sure to save it next time there's a problem.

Here is the entire java file for my "Database" class, edited to redact
the company top-level domain:

http://apaste.info/6pX

The object that I am locking on (0x00000000db0011c8) is a global lock
for my whole program which I synchronize on for a handful of
normally-short-lived db-related operations just to be sure that they
aren't happening simultaneously, and to help make sure that the outcome
of certain operations is visible across threads.  I'm reasonably sure
that there are no operations in those synchronization blocks that are
expected to be slow, though I haven't verified that to 100% certainty.
One of the operations handled in that kind of synchronization lock is
the optimization of certain DB tables, which only happens if the size of
those tables is below a certain threshold -- part of making sure that
the operation completes quickly and doesn't deadlock the program.

Here are all the jar files in my program's lib directory.  My program
uses javamail, jrobin, dbcp, and solrj:

commons-dbcp2-2.1.jar
commons-io-2.4.jar
commons-pool2-2.3.jar
httpclient-4.4.1.jar
httpcore-4.4.1.jar
httpmime-4.4.1.jar
javax.mail.jar
jcl-over-slf4j-1.7.12.jar
jrobin-1.5.14.jar
jul-to-slf4j-1.7.12.jar
log4j-1.2.17.jar
mysql-connector-java-5.1.35-bin.jar
noggit-0.7.jar
slf4j-api-1.7.12.jar
slf4j-log4j12-1.7.12.jar
solr-solrj-5.0.0.jar
stax2-api-3.1.1.jar
woodstox-core-asl-4.2.0.jar
zookeeper-3.4.6.jar

I do not have remote JMX enabled for this application, and the server
where it runs does not have a gui.  I will add some code that spits out
the three requested pool stats every time the watchdog thread detects a
problem.  I cannot predict when there will be a problem ... this code
can run for weeks with nothing going wrong, or the deadlock might happen
only a few minutes after startup.

Thanks,
Shawn


---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@commons.apache.org
For additional commands, e-mail: user-help@commons.apache.org


Mime
View raw message