jackrabbit-oak-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Michael Marth <mma...@adobe.com>
Subject Re: When optimistic locking fails
Date Fri, 08 Mar 2013 10:54:02 GMT
Jukka, Marcel,

you saw this problem for SegmentMK which uses branch-merge extensively, but is this not a
problem all distributed MK implementations will run into? After all, branch-merge is part
of the MK API. Unless the MK impl uses pessimistic locking from the start, of course.
In particular, I wonder about the MongoMK.


On Mar 7, 2013, at 11:06 AM, Jukka Zitting wrote:

> Hi,
> There are a few scenarios where the optimistic locking approach used
> by the SegmentMK fails in practice:
> 1) A large batch operation while other smaller changes are being committed.
> 2) Lots of concurrent changes being committed against the same journal.
> In scenario 1 the large operation like an import can't complete itself
> since while it is rebasing itself and re-applying commit hooks other
> smaller operations have already updated the journal, triggering new
> rounds of rebasing and hook processing for the large operation until
> it bails out with the "System overloaded" exception.
> In scenario 2 the same "System overloaded" exception occurs once there
> are too many concurrent changes for the system to keep up with when
> using just a single journal. As noted by Marcel and others, this case
> comes up pretty quickly in a benchmark that explicitly tries to push
> the system to the limit.
> While in scenario 2 the "System overloaded" exception is a valid
> alternative to a potentially prolonged wait until the commit can go
> through, in scenario 1 it is clearly troublesome. Thus I'd like to
> address it, and the solution I have in mind actually works for both
> cases:
> When encountering a case where the optimistic locking mechanism can't
> push a commit through in say one second, instead of waiting for a
> longer while I'd have the SegmentMK fall back to pessimistic locking
> where it explicitly acquired a hard lock on the journal and does the
> rebase/hook processing one more time while holding that lock. This
> guarantees that all commits will go through eventually (unless there's
> a conflict or a validation failure), while keeping the benefits of
> optimistic locking for most cases. And even for scenario 1 the bulk of
> the commit has already been persisted when the pessimistic locking
> kicks in, so the critical section should still be much smaller than
> with Jackrabbit 2.x where the lock is held also while the change set
> is being persisted.
> BR,
> Jukka Zitting

View raw message