jackrabbit-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "sbarriba" <sbarr...@yahoo.co.uk>
Subject RE: Intermittent JackRabbit dead lock with concurrent QueryImpl.execute() and NodeImpl.getProperty()
Date Tue, 24 Jun 2008 07:21:20 GMT
Hi Marcel et al,
As follow up to this issue. 
Where a server is under high load repeatedly taking thread dumps
occasionally shows threads waiting on locks that no one holds. We think
that's simply because as the number of threads increase, the level of
concurrency increases and the OS is context switching, there is a period of
time between one thread releasing and another taking the lock that the dump
implies no one is holding it.

Regards,
Shaun

-----Original Message-----
From: Marcel Reutegger [mailto:marcel.reutegger@gmx.net] 
Sent: 19 May 2008 09:59
To: users@jackrabbit.apache.org
Subject: Re: Intermittent JackRabbit dead lock with concurrent
QueryImpl.execute() and NodeImpl.getProperty()

Hi Shaun,

can you please provide the complete thread dump? thanks

regards
  marcel

sbarriba wrote:
> Hi Marcel et al,
> Just had another occurrence of the deadlock issue within JackRabbit 1.4.
> Perhaps this lock chain info will enlighten someone.
> I'm expecting that if I search the full thread dump we'd find 'something'
> reporting that its holding lock "0x438fe98" - but nothing does? Odd.
> 
> The JVM being used is BEA JRockit(R)
> R27.4.0-90-89592-1.5.0_12-20070928-1715-linux-x86_64 if that helps.
> 
> All comments welcome.
> Regards,
> Shaun
> 
> Blocked lock chains
> ===================
> 20+ threads all reporting waiting on Thread 21
> Chain 7:
> "[ACTIVE] ExecuteThread: '3' for queue: 'weblogic.kernel.Default
> (self-tuning)'" id=66 idx=0x104 tid=21527 waiting for
>
EDU/oswego/cs/dl/util/concurrent/WriterPreferenceReadWriteLock$ReaderLock@0x
> 438fe98 held by:
> "[ACTIVE] ExecuteThread: '21' for queue: 'weblogic.kernel.Default
> (self-tuning)'" id=281 idx=0x1a0 tid=23875 in chain 1
> 
> Open lock chains
> ================
> Chain 1:
> "[ACTIVE] ExecuteThread: '0' for queue: 'weblogic.kernel.Default
> (self-tuning)'" id=14 idx=0x40 tid=21444 waiting for
>
EDU/oswego/cs/dl/util/concurrent/WriterPreferenceReadWriteLock$ReaderLock@0x
> 438fe98 held by:
> "[ACTIVE] ExecuteThread: '21' for queue: 'weblogic.kernel.Default
> (self-tuning)'" id=281 idx=0x1a0 tid=23875 (active)
> 
> Chain 3:
> "ExecuteThread: '1' for queue: 'weblogic.socket.Muxer'" id=22 idx=0x60
> tid=21452 waiting for java/lang/String@0x884a6a8 held by:
> "ExecuteThread: '0' for queue: 'weblogic.socket.Muxer'" id=21 idx=0x5c
> tid=21451 (active)
> 
> 
> Stack for Thread 21 looks like:
> 
> [ACTIVE] ExecuteThread: '21' for queue: 'weblogic.kernel.Default
> (self-tuning)'" id=281 idx=0x1a0 tid=23875 prio=5 alive, in native, daemon
>     at jrockit/vm/Locks.monitorEnterUnmatched(Ljava/lang/Object;)V(Native
> Method)
>     at
>
EDU/oswego/cs/dl/util/concurrent/WriterPreferenceReadWriteLock$ReaderLock.ac
> quire()V(Unknown Source)[optimized]
>     at
>
org/apache/jackrabbit/core/state/DefaultISMLocking$ReadLockImpl.<init>(Defau
> ltISMLocking.java:103)[inlined]
>     at
>
org/apache/jackrabbit/core/state/DefaultISMLocking$ReadLockImpl.<init>(Defau
> ltISMLocking.java:97)[inlined]
>     at
>
org/apache/jackrabbit/core/state/DefaultISMLocking.acquireReadLock(DefaultIS
> MLocking.java:65)[optimized]
>     at
>
org/apache/jackrabbit/core/state/SharedItemStateManager.acquireReadLock(Shar
> edItemStateManager.java:1438)[inlined]
>     at
>
org/apache/jackrabbit/core/state/SharedItemStateManager.getItemState(SharedI
> temStateManager.java:237)[optimized]
>     at
>
org/apache/jackrabbit/core/state/LocalItemStateManager.getNodeState(LocalIte
> mStateManager.java:93)[inlined]
>     at
>
org/apache/jackrabbit/core/state/LocalItemStateManager.getItemState(LocalIte
> mStateManager.java:148)[inlined]
>     at
>
org/apache/jackrabbit/core/state/XAItemStateManager.getItemState(XAItemState
> Manager.java:226)[optimized]
>     ^-- Holding lock:
> org/apache/jackrabbit/core/state/XAItemStateManager@0x21aa13f8[thin lock]
>     at
>
org/apache/jackrabbit/core/state/SessionItemStateManager.getItemState(Sessio
> nItemStateManager.java:175)[optimized]
>     at
>
org/apache/jackrabbit/core/ItemManager.createItemInstance(ItemManager.java:5
> 64)[inlined]
>     at
>
org/apache/jackrabbit/core/ItemManager.getItem(ItemManager.java:395)[inlined
> ]
>     at
>
org/apache/jackrabbit/core/LazyItemIterator.prefetchNext(LazyItemIterator.ja
> va:90)[optimized]
>     ^-- Holding lock:
> org/apache/jackrabbit/core/ItemManager@0x21aa19c0[recursive]
>     at
>
org/apache/jackrabbit/core/LazyItemIterator.<init>(LazyItemIterator.java:75)
> [inlined]
>     at
>
org/apache/jackrabbit/core/ItemManager.getChildNodes(ItemManager.java:488)[i
> nlined]
>     at
>
org/apache/jackrabbit/core/NodeImpl.getNodes(NodeImpl.java:2501)[optimized]
>     ^-- Holding lock:
org/apache/jackrabbit/core/ItemManager@0x21aa19c0[thin
> lock]
>     at
>
javax/jcr/util/TraversingItemVisitor.visit(TraversingItemVisitor.java:177)[i
> nlined]
>     at
> org/apache/jackrabbit/core/NodeImpl.accept(NodeImpl.java:1941)[inlined]
>     at
>
org/apache/jackrabbit/core/NodeImpl.getNodes(NodeImpl.java:2787)[optimized]
> 
> 
> -----Original Message-----
> From: Marcel Reutegger [mailto:marcel.reutegger@gmx.net] 
> Sent: 09 May 2008 13:56
> To: users@jackrabbit.apache.org
> Subject: Re: Intermittent JackRabbit dead lock with concurrent
> QueryImpl.execute() and NodeImpl.getProperty()
> 
> Hi,
> 
> sbarriba wrote:
>> I just noticed
>>
>
http://www.nabble.com/-jira--Created:-(JCR-1334)-Deadlock-with-XA-enabled-td
>> 14997630.html. Could that be related given we've deployed JackRabbit as a
>> RAR module within Weblogic?
> 
> I don't think those are related, both threads only read from the
repository.
> 
>> Blocked lock chains
>> ===================
>>     Chain 30:
>>     "[ACTIVE] ExecuteThread: '55' for queue: 'weblogic.kernel.Default
>>     (self-tuning)'" id=27350 idx=0x1cc tid=26815 waiting for
>>
> EDU/oswego/cs/dl/util/concurrent/WriterPreferenceReadWriteLock$ReaderLoc
>>     k@0xbd7c718 held by:
>>     "[ACTIVE] ExecuteThread: '50' for queue: 'weblogic.kernel.Default
>> (self-tuning)'" id=27345 idx=0x220 tid=26810 in chain 1
>>
>> Chain 36:
>> "[ACTIVE] ExecuteThread: '50' for queue: 'weblogic.kernel.Default
>> (self-tuning)'" id=27345 idx=0x220 tid=26810 waiting for
>> EDU/oswego/cs/dl/util/concurrent/WriterPreferenceReadWriteLock$ReaderLoc
>> k@0xbd7c718 held by:
>> "[ACTIVE] ExecuteThread: '55' for queue: 'weblogic.kernel.Default
>> (self-tuning)'" id=27350 idx=0x1cc tid=26815 in chain 1
> 
> this is pretty strange because both threads try to acquire a read lock,
> which is 
> guaranteed to not block, unless some other thread owns the write lock.
> 
> even more strange, the JVM claims that *both* threads hold the same
monitor!
> 
> -> ReaderLock@0xbd7c718 held by: "[ACTIVE] ExecuteThread: '55' [...]
> -> ReaderLock@0xbd7c718 held by: "[ACTIVE] ExecuteThread: '50' [...]
> 
> maybe a JVM upgrade will help?
> 
> regards
>   marcel
> 
> 
> 



Mime
View raw message