db-derby-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Andreas Fredriksson <andreas.fredriks...@digitalroute.com>
Subject Class loading deadlock
Date Tue, 13 Sep 2005 07:54:46 GMT

Hey list,
we've re-targeted our system to Derby and now we're seeing some class
loading-related deadlocks.

To give some background, our system loads code from the database and
other various sources (through our own set of class loaders). This has
worked fine with Oracle, MS SQL Server and many other databases, but
with Derby there is an issue with the interaction pattern of class
loaders and the database helper objects.

The deadlock occurs when one thread has locked the classloader to find a
certain class which then triggers a database query to find the right
data in one thread. At the same time a daily query is running against
the database, which triggers class loader operations against the already
locked class loader instance.

Short and sweet:

Thread A (loading a class/resource via a ResourceBundle)
Lock ClassLoader (implicit through ClassLoader.loadClass())
Lock DB helper object (when issuing a query)

Thread B (executing some SQL query)
Lock DB helper object (to generate code?)
Lock ClassLoader (implicit via loadClass())

This is the ABC of deadlocks, but it's pretty hard to resolve because of
the synchronized ClassLoader.loadClass() method. The only option I see
is to remove the ReflectClassesJava2 lock around the classloader
operations and handle failures by retrying them. Would that work?

I've appended the full call stack information at the bottom of this
email.

What can we do to resolve this bug?

Best regards,
Andreas

-----------------------------------------------------------------------

Found one Java-level deadlock:
=============================
"Thread-97":
  waiting to lock monitor 0x080c0fcc (object 0xec053590, a
com.digitalroute.picostart.PlatformClassLoader),
  which is held by "Thread-98"
"Thread-98":
  waiting to lock monitor 0x080c1004 (object 0xebfef7c0, a
org.apache.derby.impl.services.reflect.ReflectClassesJava2),
  which is held by "Thread-97"

Java stack information for the threads listed above:
===================================================
"Thread-97":
        at java.lang.ClassLoader.loadClass(ClassLoader.java:278)
        - waiting to lock <0xec053590> (a
com.digitalroute.picostart.PlatformClassLoader)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:235)
        at
org.apache.derby.impl.services.reflect.ReflectClassesJava2.loadClassNotInDatabaseJar(Unknown
Source)
        - locked <0xebfef7c0> (a
org.apache.derby.impl.services.reflect.ReflectClassesJava2)
        at
org.apache.derby.impl.services.reflect.DatabaseClasses.loadApplicationClass(Unknown Source)
        at
org.apache.derby.iapi.services.loader.ClassInspector.getClass(Unknown
Source)
        at
org.apache.derby.iapi.services.loader.ClassInspector.accessible(Unknown
Source)
        at
org.apache.derby.impl.sql.compile.QueryTreeNode.verifyClassExist(Unknown
Source)
        at
org.apache.derby.impl.sql.compile.AggregateNode.checkAggregatorClassName(Unknown Source)
        at
org.apache.derby.impl.sql.compile.AggregateNode.bindExpression(Unknown
Source)
        at
org.apache.derby.impl.sql.compile.SQLToJavaValueNode.bindExpression(Unknown Source)
        at
org.apache.derby.impl.sql.compile.MethodCallNode.bindParameters(Unknown
Source)
        at
org.apache.derby.impl.sql.compile.StaticMethodCallNode.bindExpression(Unknown Source)
        at
org.apache.derby.impl.sql.compile.JavaToSQLValueNode.bindExpression(Unknown Source)
        at
org.apache.derby.impl.sql.compile.ResultColumn.bindExpression(Unknown
Source)
        at
org.apache.derby.impl.sql.compile.ResultColumnList.bindExpressions(Unknown Source)
        at
org.apache.derby.impl.sql.compile.SelectNode.bindExpressions(Unknown
Source)
        at
org.apache.derby.impl.sql.compile.FromSubquery.bindExpressions(Unknown
Source)
        at
org.apache.derby.impl.sql.compile.FromList.bindExpressions(Unknown
Source)
        at
org.apache.derby.impl.sql.compile.SelectNode.bindExpressions(Unknown
Source)
        at
org.apache.derby.impl.sql.compile.DMLStatementNode.bindExpressions(Unknown Source)
        at
org.apache.derby.impl.sql.compile.DMLStatementNode.bind(Unknown Source)
        at org.apache.derby.impl.sql.compile.ReadCursorNode.bind(Unknown
Source)
        at org.apache.derby.impl.sql.compile.CursorNode.bind(Unknown
Source)
        at org.apache.derby.impl.sql.GenericStatement.prepMinion(Unknown
Source)
        at org.apache.derby.impl.sql.GenericStatement.prepare(Unknown
Source)
        at
org.apache.derby.impl.sql.conn.GenericLanguageConnectionContext.prepareInternalStatement(Unknown
Source)
        at
org.apache.derby.impl.jdbc.EmbedPreparedStatement.<init>(Unknown Source)
        at
org.apache.derby.impl.jdbc.EmbedPreparedStatement20.<init>(Unknown
Source)
        at
org.apache.derby.impl.jdbc.EmbedPreparedStatement30.<init>(Unknown
Source)
        at
org.apache.derby.jdbc.Driver30.newEmbedPreparedStatement(Unknown Source)
        at
org.apache.derby.impl.jdbc.EmbedConnection.prepareStatement(Unknown
Source)
        - locked <0xeb392740> (a
org.apache.derby.impl.jdbc.EmbedConnection30)
        at
org.apache.derby.impl.jdbc.EmbedConnection.prepareStatement(Unknown
Source)
        at
com.digitalroute.jdbc.generic_sm_get_statistics.getCompressedStat(generic_sm_get_statistics.java:141)
        at
com.digitalroute.jdbc.generic_sm_get_statistics.invoke(generic_sm_get_statistics.java:120)
        at
com.digitalroute.misc.jdbc.Dispatcher.invokeOperation(Dispatcher.java:84)
        at
com.digitalroute.statistics.StatManagerImpl.compStat(StatManagerImpl.java:774)
        at
com.digitalroute.statistics.StatManagerImpl.activate(StatManagerImpl.java:758)
        at com.digitalroute.xplan.activation.Activator
$ExecutionThread.run(Activator.java:48)


"Thread-98":
        at
org.apache.derby.impl.services.reflect.ReflectClassesJava2.loadGeneratedClassFromData(Unknown
Source)
        - waiting to lock <0xebfef7c0> (a
org.apache.derby.impl.services.reflect.ReflectClassesJava2)
        at
org.apache.derby.impl.services.reflect.DatabaseClasses.loadGeneratedClass(Unknown Source)
        at
org.apache.derby.impl.services.bytecode.GClass.getGeneratedClass(Unknown
Source)
        at
org.apache.derby.impl.sql.compile.ExpressionClassBuilder.getGeneratedClass(Unknown Source)
        at
org.apache.derby.impl.sql.compile.StatementNode.generate(Unknown Source)
        at org.apache.derby.impl.sql.GenericStatement.prepMinion(Unknown
Source)
        at org.apache.derby.impl.sql.GenericStatement.prepare(Unknown
Source)
        at
org.apache.derby.impl.sql.conn.GenericLanguageConnectionContext.prepareInternalStatement(Unknown
Source)
        at
org.apache.derby.impl.jdbc.EmbedPreparedStatement.<init>(Unknown Source)
        at
org.apache.derby.impl.jdbc.EmbedPreparedStatement20.<init>(Unknown
Source)
        at
org.apache.derby.impl.jdbc.EmbedPreparedStatement30.<init>(Unknown
Source)
        at
org.apache.derby.jdbc.Driver30.newEmbedPreparedStatement(Unknown Source)
        at
org.apache.derby.impl.jdbc.EmbedConnection.prepareStatement(Unknown
Source)
        - locked <0xeb3bad30> (a
org.apache.derby.impl.jdbc.EmbedConnection30)
        at
org.apache.derby.impl.jdbc.EmbedConnection.prepareStatement(Unknown
Source)
        at
com.digitalroute.codeserver.GenericDBStorage.lookupArchiveByType(GenericDBStorage.java:739)
        at
com.digitalroute.codeserver.CodeServerImpl.lookupArchiveByType(CodeServerImpl.java:438)
        - locked <0xebe300e0> (a
com.digitalroute.codeserver.CodeServerImpl)
        at
com.digitalroute.picostart.CSMethodInvoker.lookupArchiveByType(CSMethodInvoker.java:58)
        at
com.digitalroute.picostart.CacheManager.putArchive(CacheManager.java:448)
        - locked <0xec049e40> (a
com.digitalroute.picostart.CacheManager)
        at
com.digitalroute.picostart.CacheManager.resolveCacheEntry(CacheManager.java:231)
        - locked <0xec049e40> (a
com.digitalroute.picostart.CacheManager)
        at
com.digitalroute.picostart.CacheManager.getCacheEntry(CacheManager.java:264)
        at
com.digitalroute.picostart.PicoClassLoader.tryCacheLoad(PicoClassLoader.java:89)
        at
com.digitalroute.picostart.PicoClassLoader.findClass(PicoClassLoader.java:73)
        - locked <0xec053590> (a
com.digitalroute.picostart.PlatformClassLoader)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:289)
        - locked <0xec053590> (a
com.digitalroute.picostart.PlatformClassLoader)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:235)
        at java.util.ResourceBundle.loadBundle(ResourceBundle.java:1058)
        at java.util.ResourceBundle.findBundle(ResourceBundle.java:928)
        at
java.util.ResourceBundle.getBundleImpl(ResourceBundle.java:765)
        at java.util.ResourceBundle.getBundle(ResourceBundle.java:702)
        at
com.digitalroute.devkit.misc.DRTextCode.getMessage(DRTextCode.java:140)
        at
com.digitalroute.devkit.misc.DRTextCode.<init>(DRTextCode.java:78)
        at
com.digitalroute.misc.MZTreeTextCode.TC_NAMING_LOOKUP_FAILURE(MZTreeTextCode.java:1800)
        at
com.digitalroute.misc.RMINamingHelper.lookup(RMINamingHelper.java:88)
        at
com.digitalroute.tap.manager.cleanup.CleanUpTap.initialize(CleanUpTap.java:62)
        at
com.digitalroute.devkit.task.DRTaskExecutor.initialize(DRTaskExecutor.java:38)
        at com.digitalroute.taskmanager.Task.execute(Task.java:99)
        - locked <0xec79d018> (a com.digitalroute.taskmanager.Task)
        at
com.digitalroute.taskmanager.TaskManagerImpl.activate(TaskManagerImpl.java:593)
        at com.digitalroute.xplan.activation.Activator
$ExecutionThread.run(Activator.java:48)

Found 1 deadlock.




Mime
View raw message