jackrabbit-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Robert Sauer (JIRA)" <j...@apache.org>
Subject [jira] Commented: (JCR-2554) Deadlock inside XASession on Weblogic
Date Thu, 11 Mar 2010 10:54:28 GMT

    [ https://issues.apache.org/jira/browse/JCR-2554?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12844010#action_12844010

Robert Sauer commented on JCR-2554:

Essentially yes. I had breakpoint on all critical places, and this one was trigegred.

I assume the call sequences is as follows (having the JCA adapter driven by a WLS workmanager
1. thread-a invokes prepare on behalf of XID_1
  - the write lock is granted to thread-a
  - XID_1 is stored as the ActiveXID
2. thread-a now invokes prepare on behalf of XID_2
  - tries to acquire the write lock, theoretically it should block until XID_1 was committed
  - but due to the previous association between the write lock and thread-a locking succeeds
  - thread-a now tries to set the ActiveXID to XID_2, but as XID_1 is still active, it will
just trace a warning
3. thread-b invokes commit on behalf of XID_1
  - the ActiveXID is cleared
  - the write lock gets released
4. eventually some thread invokes commit on behalf of XID_2, but as the ActiveXID was not
set, signalling readers fails

I did not know how to easily fix 2. as the lock-to-thread association seems to be fundamentally
wrong in this case. So I started with the different approach contained in the patch.

> Deadlock inside XASession on Weblogic
> -------------------------------------
>                 Key: JCR-2554
>                 URL: https://issues.apache.org/jira/browse/JCR-2554
>             Project: Jackrabbit Content Repository
>          Issue Type: Bug
>          Components: jackrabbit-core
>    Affects Versions: 1.6.1, 2.0.0
>         Environment: Weblogic 9.2
>            Reporter: Robert Sauer
>            Assignee: Claus Köll
>            Priority: Critical
>         Attachments: ConcurrentReaders.java, jr-core-xid-aware-ism-locking1.patch
> In one of our client deployments on WebLogic 9.2 we observed JackRabbit sessions going
stale in a load test. This was observed against release 1.6.1 (to which we migrated due to
concurrency related issues JCR-2081 and JCR-2237). Same effect with 2.0.0.
> I could finally reproduce this issue locally. And it seems to boil down to WLS invoking
the sequence of <prepare> ... <release> ... <commit> on one XA session from
multiple threads, as it seems breaking assumptions of the thread-bound java.util.concurrent-RWLock
based DefaultISMLocking class.
> Effectively the setActiveXid(..) method on DefaultISMLocking$RWLock fails as the old
active XID was not yet cleared. With the result of more and more sessions deadlocking in below's
invocation stack.
> {code}
> "[ACTIVE] ExecuteThread: '27' for queue: 'weblogic.kernel.Default (self-tuning)'" daemon
prio=1 tid=0x33fc3ec0 nid=0x2324 in Object.wait() [0x2156a000..0x2156beb0] at java.lang.Object.wait(Native
Method) - waiting on <0x68a54698> (a EDU.oswego.cs.dl.util.concurrent.WriterPreferenceReadWriteLock$WriterLock)
at java.lang.Object.wait(Object.java:474) at EDU.oswego.cs.dl.util.concurrent.WriterPreferenceReadWriteLock$WriterLock.acquire(Unknown
Source) - locked <0x68a54698> (a EDU.oswego.cs.dl.util.concurrent.WriterPreferenceReadWriteLock$WriterLock)
at org.apache.jackrabbit.core.state.DefaultISMLocking$1.<init>(DefaultISMLocking.java:64)
at org.apache.jackrabbit.core.state.DefaultISMLocking.acquireWriteLock(DefaultISMLocking.java:61)
at org.apache.jackrabbit.core.version.AbstractVersionManager.acquireWriteLock(AbstractVersionManager.java:146)
at org.apache.jackrabbit.core.version.XAVersionManager$1.prepare(XAVersionManager.java:562)
at org.apache.jackrabbit.core.TransactionContext.prepare(TransactionContext.java:154) - locked
<0x6dc2ad88> (a org.apache.jackrabbit.core.TransactionContext) at org.apache.jackrabbit.core.XASessionImpl.prepare(XASessionImpl.java:331)
at org.apache.jackrabbit.jca.TransactionBoundXAResource.prepare(TransactionBoundXAResource.java:68)
at weblogic.connector.security.layer.AdapterLayer.prepare(AdapterLayer.java:397) at weblogic.connector.transaction.outbound.XAWrapper.prepare(XAWrapper.java:297)
at weblogic.transaction.internal.XAServerResourceInfo.prepare(XAServerResourceInfo.java:1276)
at weblogic.transaction.internal.XAServerResourceInfo.prepare(XAServerResourceInfo.java:499)
at weblogic.transaction.internal.ServerSCInfo$1.execute(ServerSCInfo.java:335) at weblogic.kernel.Kernel.executeIfIdle(Kernel.java:243)
at weblogic.transaction.internal.ServerSCInfo.startPrepare(ServerSCInfo.java:326) at weblogic.transaction.internal.ServerTransactionImpl.localPrepare(ServerTransactionImpl.java:2516)
at weblogic.transaction.internal.ServerTransactionImpl.globalPrepare(ServerTransactionImpl.java:2211)
at weblogic.transaction.internal.ServerTransactionImpl.internalCommit(ServerTransactionImpl.java:266)
at weblogic.transaction.internal.ServerTransactionImpl.commit(ServerTransactionImpl.java:227)
at weblogic.transaction.internal.TransactionManagerImpl.commit(TransactionManagerImpl.java:283)
at org.springframework.transaction.jta.JtaTransactionManager.doCommit(JtaTransactionManager.java:1028)
at org.springframework.transaction.support.AbstractPlatformTransactionManager.processCommit(AbstractPlatformTransactionManager.java:709)
at org.springframework.transaction.support.AbstractPlatformTransactionManager.commit(AbstractPlatformTransactionManager.java:678)
> {code}

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message