jackrabbit-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From <Robert.Sa...@sungard.com>
Subject Deadlock inside XASession on Weblogic (JR 1.6.1)
Date Tue, 09 Mar 2010 18:47:54 GMT
Hi,
 
in one of our client deployments on WebLogic 9.2 we observed JackRabbit
sessions going stale in a load test. This was observed against release
1.6.1 (to which we migrated due to concurrency related issues JCR-2081
and JCR-2237). Same effect with 2.0.0.
 
I could finally reproduce this issue locally. And it seems to boil down
to WLS invoking the sequence of <prepare> ... <release> ... <commit> on
one XA session from multiple threads, as it seems breaking assumptions
of the thread-bound java.util.concurrent-RWLock based DefaultISMLocking
class.
Effectively the setActiveXid(..) method on DefaultISMLocking$RWLock
fails as the old active XID was not yet cleared. With the result of more
and more sessions deadlocking in below's invocation stack.
 
Is this a known issue already? Any existing workarounds?
 
The test driver is relatively simple as it just involves multiple
threads reading from the repository in parallel sessions. Downside is
just that I'm currently using an in house developed library on top of
JCR which unfortuantely I cannot pass around. But I think I can come up
with an isolated test driver that talks just plain JCR.
 
In addition, I created an implementation of ISMLocking which associates
locks directly with Xids instead of using thread-bound
java.util.concurrent-locks. This seems to work pretty well so far in
this isolated test case. I can now read with 45 concurrent threads
without locking up sessions.
My question is what will be the best way to carry this forward.
Unfortunately in class AbstractVersionManagerImpl there is a hardcoded
use of DefaultISMLocking.
 
* Would it make sense to have a configuration parameter for which
ISMLocking to use for the rwLock member in AbstractVersionManager?
* May be a better approach could be to inject an appropriate locking
implementation from XAVersionManager as this will provide a means to
perfectly adjust to an XA environment?
* Is there some interest in reviewing my implementation and may be
including it or a derivative into the code base?
 
Looking forward to any feedback.
 
Best regards
Robert
 
[snip]
"[ACTIVE] ExecuteThread: '27' for queue: 'weblogic.kernel.Default
(self-tuning)'" daemon prio=1 tid=0x33fc3ec0 nid=0x2324 in Object.wait()
[0x2156a000..0x2156beb0]
 at java.lang.Object.wait(Native Method)
 - waiting on <0x68a54698> (a
EDU.oswego.cs.dl.util.concurrent.WriterPreferenceReadWriteLock$WriterLoc
k)
 at java.lang.Object.wait(Object.java:474)
 at
EDU.oswego.cs.dl.util.concurrent.WriterPreferenceReadWriteLock$WriterLoc
k.acquire(Unknown Source)
 - locked <0x68a54698> (a
EDU.oswego.cs.dl.util.concurrent.WriterPreferenceReadWriteLock$WriterLoc
k)
 at
org.apache.jackrabbit.core.state.DefaultISMLocking$1.<init>(DefaultISMLo
cking.java:64)
 at
org.apache.jackrabbit.core.state.DefaultISMLocking.acquireWriteLock(Defa
ultISMLocking.java:61)
 at
org.apache.jackrabbit.core.version.AbstractVersionManager.acquireWriteLo
ck(AbstractVersionManager.java:146)
 at
org.apache.jackrabbit.core.version.XAVersionManager$1.prepare(XAVersionM
anager.java:562)
 at
org.apache.jackrabbit.core.TransactionContext.prepare(TransactionContext
.java:154)
 - locked <0x6dc2ad88> (a org.apache.jackrabbit.core.TransactionContext)
 at
org.apache.jackrabbit.core.XASessionImpl.prepare(XASessionImpl.java:331)
 at
org.apache.jackrabbit.jca.TransactionBoundXAResource.prepare(Transaction
BoundXAResource.java:68)
 at
weblogic.connector.security.layer.AdapterLayer.prepare(AdapterLayer.java
:397)
 at
weblogic.connector.transaction.outbound.XAWrapper.prepare(XAWrapper.java
:297)
 at
weblogic.transaction.internal.XAServerResourceInfo.prepare(XAServerResou
rceInfo.java:1276)
 at
weblogic.transaction.internal.XAServerResourceInfo.prepare(XAServerResou
rceInfo.java:499)
 at
weblogic.transaction.internal.ServerSCInfo$1.execute(ServerSCInfo.java:3
35)
 at weblogic.kernel.Kernel.executeIfIdle(Kernel.java:243)
 at
weblogic.transaction.internal.ServerSCInfo.startPrepare(ServerSCInfo.jav
a:326)
 at
weblogic.transaction.internal.ServerTransactionImpl.localPrepare(ServerT
ransactionImpl.java:2516)
 at
weblogic.transaction.internal.ServerTransactionImpl.globalPrepare(Server
TransactionImpl.java:2211)
 at
weblogic.transaction.internal.ServerTransactionImpl.internalCommit(Serve
rTransactionImpl.java:266)
 at
weblogic.transaction.internal.ServerTransactionImpl.commit(ServerTransac
tionImpl.java:227)
 at
weblogic.transaction.internal.TransactionManagerImpl.commit(TransactionM
anagerImpl.java:283)
 at
org.springframework.transaction.jta.JtaTransactionManager.doCommit(JtaTr
ansactionManager.java:1028)
 at
org.springframework.transaction.support.AbstractPlatformTransactionManag
er.processCommit(AbstractPlatformTransactionManager.java:709)
 at
org.springframework.transaction.support.AbstractPlatformTransactionManag
er.commit(AbstractPlatformTransactionManager.java:678)
[/snip]

Mime
View raw message