Mailing-List: contact oak-dev-help@jackrabbit.apache.org; run by ezmlm
Precedence: bulk
Reply-To: oak-dev@jackrabbit.apache.org
Received-SPF: pass (nike.apache.org: domain of jukka.zitting@gmail.com
 designates 209.85.219.47 as permitted sender)
MIME-Version: 1.0
From: Jukka Zitting <jukka.zitting@gmail.com>
Date: Thu, 7 Mar 2013 12:06:16 +0200
Message-ID: 
 <CAOFYJNYaF-rk0tuuOtPAZWFYS7Q3A5wPvc9Hs+Kqn=t3_BqZRA@mail.gmail.com>
Subject: When optimistic locking fails
To: Oak devs <oak-dev@jackrabbit.apache.org>
Content-Type: text/plain; charset=ISO-8859-1

Hi,

There are a few scenarios where the optimistic locking approach used
by the SegmentMK fails in practice:

1) A large batch operation while other smaller changes are being committed.

2) Lots of concurrent changes being committed against the same journal.

In scenario 1 the large operation like an import can't complete itself
since while it is rebasing itself and re-applying commit hooks other
smaller operations have already updated the journal, triggering new
rounds of rebasing and hook processing for the large operation until
it bails out with the "System overloaded" exception.

In scenario 2 the same "System overloaded" exception occurs once there
are too many concurrent changes for the system to keep up with when
using just a single journal. As noted by Marcel and others, this case
comes up pretty quickly in a benchmark that explicitly tries to push
the system to the limit.

While in scenario 2 the "System overloaded" exception is a valid
alternative to a potentially prolonged wait until the commit can go
through, in scenario 1 it is clearly troublesome. Thus I'd like to
address it, and the solution I have in mind actually works for both
cases:

When encountering a case where the optimistic locking mechanism can't
push a commit through in say one second, instead of waiting for a
longer while I'd have the SegmentMK fall back to pessimistic locking
where it explicitly acquired a hard lock on the journal and does the
rebase/hook processing one more time while holding that lock. This
guarantees that all commits will go through eventually (unless there's
a conflict or a validation failure), while keeping the benefits of
optimistic locking for most cases. And even for scenario 1 the bulk of
the commit has already been persisted when the pessimistic locking
kicks in, so the critical section should still be much smaller than
with Jackrabbit 2.x where the lock is held also while the change set
is being persisted.

BR,

Jukka Zitting