Mailing-List: contact dev-help@directory.apache.org; run by ezmlm
Precedence: bulk
Reply-To: "Apache Directory Developers List" <dev@directory.apache.org>
Received-SPF: pass (athena.apache.org: domain of elecharny@gmail.com
 designates 74.125.82.44 as permitted sender)
DomainKey-Signature: a=rsa-sha1; c=nofws;
        d=gmail.com; s=gamma;
        h=message-id:date:from:reply-to:user-agent:mime-version:to:subject
         :content-type:content-transfer-encoding;
        b=lT555E8jrNKk3F+IkxkqKavotRRBAsxh3jrN41Xc7WS3yvjLEqy6rKsqM4cKsLl9pX
         KZFbNIMNV3cjSScvhbXpyHVa6rWIL9baqKALysNEoi9/EOY/QfUhtyws7ERT0meP1E/i
         mLi0MbUW3YMFq7acwjx1/BPPrNdEJLWcD9jNQ=
Message-ID: <4D05F876.2040000@gmail.com>
Date: Mon, 13 Dec 2010 11:41:58 +0100
From: Emmanuel Lecharny <elecharny@gmail.com>
Reply-To: elecharny@apache.org
User-Agent: Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10.6; en-US;
 rv:1.9.2.13) Gecko/20101207 Thunderbird/3.1.7
MIME-Version: 1.0
To: Apache Directory Developers List <dev@directory.apache.org>
Subject: AdministrativePoint handling
Content-Type: text/plain; charset=ISO-8859-1; format=flowed
Content-Transfer-Encoding: 8bit

Hi guys,

I'm resuming the APs implementation. I have had a new idea this morning 
about the best way to handle them.

Currently, we had devised about too possible options

1) Processing the entries while adding a new AP
The idea is that when we add a new AP (or delete an existing one, or 
modifying an existing one), we evaluate all the underlying entries and 
update each one which is selected by the associated SubtreeSpecification 
by adding a reference to its associated AP in the entry.
This is done once, then when a user grab an entry, we don't need to 
evaluate it again, as this work has already been done, so we accept to 
pay the price once, but it's free for every operation after this initial 
update.

The price to pay is pretty expensive, if we consider a huge base with 
hundred of thousands entries, as we have to update many, and as an 
update is slow (we currently can process 600 per second of such updates 
- depending on the server we are running on, of course -, so for 1 000 
000 entries it might take up to 30 minutes ).

2) Processing the evaluation on the fly
The second idea is to evaluate the entry when a user requests it, to 
avoid paying the price for the initial update.

The big problem with this approach is that if we have many APs 
SubtreeSpecifications to evaluate, this evaluation is not free, plus it 
has to occur *everytime* a user access the entry. That could slow down 
the server a lot (probably double the time it takes to return an entry).

So far, we have decider to go for option #1, simply because adding an AP 
is an administrative task, which does not occur very often. We all 
realize that it has drawbacks, but it's far better than option #2 anyway.

Now, I'd like to propose a third option, which might be acceptable, 
considering that we will pay a small price compared to both option #1 
and #2.

3) Processing on the fly and store the result
The idea is to process the evaluation when someone fetches an entry, but 
to store the evaluation result in the entry. If the fetched entry has 
already been evaluated, then no need to process it again.

The direct benefit will be that we won't have this huge initial update 
required by option #1, and the entry evaluation wil be just a matter to 
check if an AT has been set instead of fully evalute the entry as in 
option #2.

How will it works ? Simple : when an AP is added, a Timestamp (TS) is 
updated. This TS will be incremented every time we add a new AP. We 
don't update any entry.
Then when a user fetch some entries, for every one of the selected 
entries, we check if it has a TS AT present. If not, then we evaluate 
the entry against the AP it depends upon, and when done, we add the last 
TS into the entry, save it and return (or not) the entry to the user.
If the entry has a TS AT, then we have two cases
- The TS equals the last TS : the entry has already been evaluated, and 
we can return it (or not, depending on the evaluation result)
- The TS is below, so we reevaluate the entry.

Now, how will we distinguish between an entry evaluating to true or 
false ? Easy : if the entry is selected by a SubtreeSpecification, we 
will have a ref to the associated AP in it, otherwise no.

I do think that this third option has great advantages, and not only 
from the user or admin POV : the implementation will be greatly 
improved, as we don't have to deal with all the cases when handling an 
AP addition/modification/deletion.

One last thing : one may object that the on the fly evaluation is not 
acceptable, as it will cost a lot for the first access to an entry. 
True. But I claim that it's the administrator to deal with this problem, 
and then its not anymore a user issue. When the admin add an new AP, he 
also can fetch all the entries below the added AP, and they will be 
updated on the fly. The users won't be impacted anymore.

thoughts ?

-- 
Regards,
Cordialement,
Emmanuel L�charny
www.iktek.com