felix-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Felix Meschberger <fmesc...@adobe.com>
Subject Re: "Deadlock" on Felix.acquireBundleLock
Date Tue, 14 Feb 2012 08:51:33 GMT

Am 13.02.2012 um 15:43 schrieb Richard S. Hall:

> On 2/13/12 06:06 , Felix Meschberger wrote:
>> Hi,
>> I recently got a number of stacktraces, where a variable number of threads is blocked
on "m_bundleLock.wait()" inside Felix.acquireBundleLock. From pure code inspection, it should
not possible for the framework to stall there.
>> It seems that Guillaume Nodet also hit this problem and added code to throw an IllegalStateException
if the wait() call is interrupted (FELIX-2784 [1]).
>> Other times I hit a similar issue in the Felix.acquireGlobalLock method for which
I reported FELIX-3067 [2].
>> It looks like both lock acquisition methods are prone to some kind of deadlock. The
acquireGlobalLock method has the ability to fail by reporting such failure with a return code.
The acquireBundleLock has no ability to fail (except throwing an exception).
>> I think similar to my FELIX-3067 proposal the acquireBundleLock method should only
wait a limited time and then retry acquiring the lock. Only if this fails for a number of
times, acquisition should be aborted and the method fail with an IllegalStateException.
>> WDYT ?
> The biggest issue I have is that these sorts of fixes simply address the 
> symptoms and may make it more difficult to figure out the root cause. 
> Understandably, these issues are notoriously difficult to reproduce...

Couldn't agree more.

> Conceptually, the code should not allow deadlocks between bundle lock 
> acquirers and global lock acquirers, much like readers and writers. The 
> global lock (i.e., write lock) cannot be acquired if there are any 
> bundle lock holders (i.e., read lock).

Yes, but I have had a situation where three threads (FelixStartLevel starting bundles, FelixPackageAdmin
refreshing bundles and an installer thread installing bundles (and configuration)). All three
were contending for the same bundle lock without one of these obviously holding the global

The result was a complete stand-still.

I will try to reproduce this issue and find out and come back with my findings.


> Bundle lock operations should only impact one bundle at time, of course, 
> this isn't always the case since synchronous event delivery introduces 
> some difficult here. All multiple bundle operations (e.g., resolves, 
> refreshes) must acquire the global. The only difficulty is that 
> sometimes you start off with a bundle lock and need to acquire the 
> global lock later because you didn't know in advance you would need the 
> global lock (e.g., starting a bundle and then determining you need to 
> resolve it or for resolving dynamic imports), in which case the bundle 
> lock needs to be promoted to a global lock, which leads to some 
> confusing code about trying to determine if there are cycles among 
> bundles trying to promote bundle locks.
> Yeah, it's confusing and complicated, but still it would be better to 
> get a somewhat reproducible case and try to figure out if there is a 
> better solution than just introducing more error cases...
> But if that's not possible, then we can do what we have to do...
> -> richard
>> Regards
>> Felix
>> PS: We are still running 3.0.8 but the acquireBundleLock and acquireGlobalLock are
the same as in trunk (except for Guillaume's FELIX-2784 fix).
>> [1] https://issues.apache.org/jira/browse/FELIX-2784
>> [2] https://issues.apache.org/jira/browse/FELIX-3067

View raw message