db-derby-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Bryan Pendleton (JIRA)" <j...@apache.org>
Subject [jira] Updated: (DERBY-666) Enhance derby.locks.deadlockTrace to print stack traces for all threads involved in a deadlock
Date Wed, 11 Jul 2007 23:54:04 GMT

     [ https://issues.apache.org/jira/browse/DERBY-666?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel

Bryan Pendleton updated DERBY-666:

    Attachment: experimental_jdk_1_5.diff

Attached is experimental_jdk_1_5.diff, which may be useful to others.


This patch contains JDK 1.5-specific code, copied almost verbatim
from the comments above, to use Thread.getAllStackTraces() to include
complete JVM stack trace information in the deadlock exception.

To build and run with this code, here's what I did:
1) Built a vanilla copy of the trunk using the standard (JDK 1.4-based) build environment
2) Applied the patch to create a modified Deadlock.java
3) Compiled *just* Deadlock.java using JDK 1.5 using the command:

javac -d classes -classpath classes -sourcepath java/engine -g java/engine/org/apache/derby/impl/services/locks/Deadlock.java

4) Then, if you are running using JAR files, you can run 'ant buildjars' to build the jars.
5) When you run your Derby application, you need to run with JDK 1.5, since Deadlock.class
has been build with JDK 1.5 and requires JDK 1.5.

By using the attached Repro.java program, I verified that the deadlock exception which is
provoked by the test program does indeed contain the stack traces of all the threads in the
JVM at the instant that the deadlock is detected.

Running the test program, first, is a good way to verify that you've built the patch correctly
and have your JDK 1.5 environment operational. When you run the test program you should get
a deadlock exception and the exception text should contain a set of stack traces that makes
sense (i.e., they show the Thread1 and Thread0 from the test program involved in the deadlock).

Assuming that you are running in an Embedded environment, so that the stack traces will be
meaningful, and can run with JDK 1.5, in order to use this code, this should be a useful tool
for diagnosing application deadlocks.

I'd be glad to help explain this process further, if these steps aren't clear.

> Enhance derby.locks.deadlockTrace to print stack traces for all threads involved in a
> ----------------------------------------------------------------------------------------------
>                 Key: DERBY-666
>                 URL: https://issues.apache.org/jira/browse/DERBY-666
>             Project: Derby
>          Issue Type: Improvement
>          Components: Store
>    Affects Versions:
>            Reporter: Bryan Pendleton
>            Priority: Minor
>         Attachments: experimental_jdk_1_5.diff, repro.java
> I was reading http://www.linux-mag.com/content/view/2134/ (good article, btw!), and it
> >   The next two properties are needed to diagnose concurrency (locking and deadlock)
> >
> >      *derby.locks.monitor=true logs all deadlocks that occur in the system.
> >      *derby.locks.deadlockTrace=true log a stack trace of all threads involved in
lock-related rollbacks.
> It seems, that, in my environment, the deadlockTrace property does not log a stack trace
of *all* threads involved in the deadlock.
> Instead, it only logs a stack trace of the *victim* thread involved in the deadlock.
> I think it would be very useful if the derby.locks.deadlockTrace setting could in fact
log a stack trace of all involved threads.
> In a posting to derby-dev, Mike Matrigali noted that an earlier implementation of a similar
feature had to be removed because it was too expensive in both time and space, but he suggested
that there might be several possible ways to implement this in an acceptably efficient manner:
> > A long time ago there use to be room in each lock to point at a
> > stack trace for each lock, but that was removed to optimize the size
> > of the lock data structure which can have many objects outstanding.
> > And creating and storing the stack for every lock was incredibly slow
> > and just was not very useful for any very active application.  I think
> > I was the only one who ever used it.
> >
> > The plan was sometime to add a per user data structure which could be
> > filled in when it was about to wait on a lock, which would give most of what is
interesting in a deadlock.
> > 
> > The current deadlockTrace is meant to dump the lock table out to derby.log when
a deadlock is encountered.
> > 
> > I agree getting a dump of all stack traces would be very useful, and
> > with the later jvm debug interfaces may now be possible - in earlier
> > JVM's there weren't any java interfaces to do so.  Does anyone have
> > the code to donate to dump all thread stacks to a buffer?
> Mike also suggested a manual technique as a workaround; it would be useful to put this
into the documentation somewhere, perhaps on the page which documents derby.locks.deadlockTrace?
Here's Mike's suggestion:
> > What I do if I can reproduce easily is set try to catch the wait by
> > hand and then depending on the environment either send the magic
> > signal or hit ctrl-break in the server window which will send the JVM
> > specific thread dumps to derby.log.
> The magic signal, btw, is 'kill -QUIT', at least with Sun JVMs in my experience.

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message