Mailing-List: contact dev-help@directory.apache.org; run by ezmlm
Precedence: bulk
Reply-To: "Apache Directory Developers List" <dev@directory.apache.org>
Received-SPF: pass (nike.apache.org: domain of akarasulu@gmail.com designates
 209.85.216.171 as permitted sender)
DomainKey-Signature: a=rsa-sha1; c=nofws;
        d=gmail.com; s=gamma;
        h=mime-version:sender:in-reply-to:references:date
         :x-google-sender-auth:message-id:subject:from:to:content-type
         :content-transfer-encoding;
        b=qavV4A7NoMQ4zwMagzDLiZ2M8NY/aScjCivWpwEMw3Wj6eCP34X8Oe9+4L4RRwWNK/
         GHc28rgsSCMmIuTXa0Hxsa+OI+XC5pFqbVkMwJkgHqPO8Q5soz8lJ4bHeckUMhbzL8HQ
         y2aR+CaZlU6r9gl9InEdf7jwIWRXpG04cm4cg=
MIME-Version: 1.0
Sender: akarasulu@gmail.com
In-Reply-To: <4D31A4C5.2050403@apache.org>
References: <4D317E82.3050901@gmail.com>
	<AANLkTi=bRn_tzf__2=P2Z-qLy90sy+ZP0n6gsCUesF7u@mail.gmail.com>
	<4D31A4C5.2050403@apache.org>
Date: Sun, 16 Jan 2011 12:52:43 +0200
Message-ID: <AANLkTik1-GgUiDcSuxA1TFrDOeAkgS5gLByx7Bb++zw5@mail.gmail.com>
Subject: Re: Some more thoughts about AP handling
From: Alex Karasulu <akarasulu@apache.org>
To: Apache Directory Developers List <dev@directory.apache.org>,
 elecharny@apache.org
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable

On Sat, Jan 15, 2011 at 3:44 PM, Emmanuel L=E9charny <elecharny@apache.org>=
 wrote:
> On 1/15/11 1:59 PM, Alex Karasulu wrote:
>>
>> On Sat, Jan 15, 2011 at 1:01 PM, Emmanuel Lecharny<elecharny@gmail.com>
>> =A0wrote:
>>>
>>> Hi guys,
>>>
>>> I'd like to share some thoughts I have had yesterday and this morning
>>> while
>>> I was lying in my bed (one of the best place to find solutions to
>>> problems
>>> with the shower).
>>>
>>> AP support suppose you deal with some constraints, as explicited in
>>> various
>>> RFCs :
>>> - we must support SAP and IAP, otherwise it's too complicated to manage
>>> ACIs
>>> when they are overlapping rights
>>> - Subentries must be referenced in selected entries as a DN
>>> - we must cover at least 3 administrative role : ACI,
>>> Collective-Attributes,
>>> Subschema, two of those aspects being described in RFCs
>>> - every operation must be atomic, ie either execute *fully* or be abort=
ed
>>> *fully*. We can't have a partial execution.
>>>
>>> In order to correctly implement APs, we don't have many possible
>>> solutions.
>>> Let me list them again :
>>> #0 don't support AP at all. Well, it's not really an option :)
>>> #1 evaluate entries on the fly when we access them, all the time
>>> #2 evaluate entries on the fly the first time we access them, and store
>>> the
>>> result
>>> #3 evaluate entries when the AP is modified, and persist the state
>>>
>>> All the 3 last approaches have pros and cons. I have ruled out option #=
0,
>>> so
>>> let's analyze the three others
>>>
>>> #1 pros :
>>> - atomicity is easy to enforce : we always do one single modification
>>> when
>>> managing APs
>>> - it's easy to implement, as we just have to handle subentries in the
>>> search
>>> and lookup methods
>>> - we don't have any operation that could impact the whole set of entrie=
s
>>
>> Also must ask, when search touches entries and updates them in this
>> lazy update mechanism, do we replicate the changes or these are
>> changes that never get replicated? I would think the later.
>
> This is #2. In #1, the entries are not modified at all on disk.
>>>
>>> #1 cons :
>>> - we will have to pay the price of the entry evaluation for every singl=
e
>>> read we do on any entry
>>
>> Every single first read correct?
>
> correct.
> <snip/>
>>>
>>> , no matter what, before we can implement either #2 or
>>> #3. And once we get this feature added in the server, then I do think
>>> that
>>> #3 is the best possible option.
>>
>> Yes yes yes ! I totally agree.
>>
>> Some things really cannot progress unless some supporting
>> infrastructure is laid down. The AP subsystem is exactly suffering
>> from this. Everything must evolve together. Otherwise we wind up
>> implementing a lot of forced solutions that are less than optimal with
>> hacks, work arounds and one offs. Things get more complex and brittle
>> causing more pain down the line.
>>
>> Let's switch into a mode of cleaning up the shop after all this heavy
>> sawing and hammering. Sit down and plan a micro strategy with a clear
>> mind and clean shop. Then knock this out in a solid and easy way.
>>
>> The AP thing was getting way too hard. I commend you on your effort,
>> and you're close but it's almost impossible to do without a local
>> transaction manager. Definitely impossible to do RIGHT without one.
>
> The main issue here is that I was trying to catch many birds with one sto=
ne.
> Not wanting to implement a transaction manager made #2 solution the only =
one
> available, but it's not a perfect one.
>
> Right now, I really think we should go step by step, following the follow=
ing
> path (roughly) :

I recommend taking an agile approach to the items below with 2-3 week
sprints each followed by a milestone release.

> - first make the current trunk working. We have an issue as the subentrie=
s
> are not read back when the server is stopped, so we can't have the ACI
> working again. That's a stupid error which can be fixed easily.
> - implement #1 : no more modifications made on the backend, all is evalua=
ted
> on the fly

After this I suggest we clean up shared, and get Studio depending
directly on it's OSGi bundles, and do some M1 releases. This can be
the first 2-3 week sprint.

> - have a TransactionManager (TM) implemented

Before this point we can cleanup some of the ADS SPIs and then
refactor the interceptor chain making transaction management a core
function of the chain rather than an interceptor. There should be no
degrees of freedom with this feature since it's a must have.

The two journals (CL & Journal) should be consolidated into a single
log even if two separate tactics are used inside the same component.
This means the CL and Journal are merged and no longer managed using
Interceptors. There are other cleanup we can do to simplify the chain
as well.

TxN management although not imparting ACID capabilities in this round,
should be clear, clean, and simple with all scaffolding erected so the
API can be properly used. Slowly we can then add all the
implementation details needed to implement all ACID features with
coming milestones.

Then a set of M2 releases can be made.

> - add support for IAPs
> - once all those pieces have been added, then we can move to #3

This can be an M3 release.

> Of course, we have other things on our plate which can be done in paralle=
l,
> like refactoring the APIs.

There will probably be incremental steps here. For M1 we can do what
seems reasonable and clear to us. Like a base line push, but I am sure
we'll begin to see more refactorings condense as we progress on each
sprint.

--=20
Alex Karasulu
My Blog :: http://www.jroller.com/akarasulu/
Apache Directory Server :: http://directory.apache.org
Apache MINA :: http://mina.apache.org
To set up a meeting with me: http://tungle.me/AlexKarasulu