directory-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Alex Karasulu <>
Subject Re: Some more thoughts about AP handling
Date Sat, 15 Jan 2011 12:59:41 GMT
On Sat, Jan 15, 2011 at 1:01 PM, Emmanuel Lecharny <> wrote:
> Hi guys,
> I'd like to share some thoughts I have had yesterday and this morning while
> I was lying in my bed (one of the best place to find solutions to problems
> with the shower).
> AP support suppose you deal with some constraints, as explicited in various
> RFCs :
> - we must support SAP and IAP, otherwise it's too complicated to manage ACIs
> when they are overlapping rights
> - Subentries must be referenced in selected entries as a DN
> - we must cover at least 3 administrative role : ACI, Collective-Attributes,
> Subschema, two of those aspects being described in RFCs
> - every operation must be atomic, ie either execute *fully* or be aborted
> *fully*. We can't have a partial execution.
> In order to correctly implement APs, we don't have many possible solutions.
> Let me list them again :
> #0 don't support AP at all. Well, it's not really an option :)
> #1 evaluate entries on the fly when we access them, all the time
> #2 evaluate entries on the fly the first time we access them, and store the
> result
> #3 evaluate entries when the AP is modified, and persist the state
> All the 3 last approaches have pros and cons. I have ruled out option #0, so
> let's analyze the three others
> #1 pros :
> - atomicity is easy to enforce : we always do one single modification when
> managing APs
> - it's easy to implement, as we just have to handle subentries in the search
> and lookup methods
> - we don't have any operation that could impact the whole set of entries

Also must ask, when search touches entries and updates them in this
lazy update mechanism, do we replicate the changes or these are
changes that never get replicated? I would think the later.

> #1 cons :
> - we will have to pay the price of the entry evaluation for every single
> read we do on any entry

Every single first read correct?

> #2 pros :
> - atomicity is somehow easier to enforce, but we still have some case where
> we have to update more than one entry when managing APs
> - potentially let the system in a consistent state if it crashes

Yeah this is making #2 look like a very poor option. I'd call it a con
rather than a pro but I think you meant this.

> #2 cons :
> - very complex to implement
> - there are some cases (like for the rename/move operations) where we either
> decide to impact a set of entries for nothing, or we have to immediately
> update a set of entries, defeating one of the [pros]
> - the first reader pay the price

Screw option #2 I see now why it's really worse than even #0.

> #3 pros :
> - the time it costs to update the entries is paid only once
> - update only the set of impacted entries
> #3 cons :
> - atomicity is not enforced, if the server crashes in the middle of an
> operation, it's unstable
> - it can take forever to update a huge base, assuming that we have to update
> all the entries, and it make take a lot of time at 1000 entries written per
> second
> - implementation is complex, but less than for #2

Inconsistency risk is something that we cannot take. #2 is the best
option or #0 I say. Better not to do if for now if we cannot do #2.

> At this point, I'm facing some big issue... If I want the rename/move
> operations implemented for option #2, it will imply either that we might
> impact many more entries than necessary, as I have to tell the parent AP
> that it's not up to date, *or* I have to adopt the #3 solution and update
> immediately the children entries...
> Ok, let's face the facts : maybe option #1 is the only possible solution we
> have atm, from different aspects :
> - delay : implementing option #1 is probably a matter of one week

Look if we make time a big deal and pump out a half assed solution
we're screw ourselves and our users. It will require more maintenance
and upkeep while not reaching our mark. I know you are of the same

Forcing a fit right now is just going to result in something bad.
We're better off leaving this as is than trying to force fit something
that is not really that valuable. Why am I thinking this way?

(1) This is an optimization but we combined it with the extra IAP
feature which to date never got used.
(2) This is about administration which is rarely ever done.
(3) Until tooling appears to enable users these features are really
not going to hit mass utilization.

So why force something now that you yourself don't like the way we
have to compromise.

> - atomicity : it's the only solution that guarantees it

Yep, but we don't have to implement full atomicity but only the needed
scaffolding to be able to make the CL behave appropriately.

> The price to pay is a slower server for search operations, as each entry
> will be evaluated again and again.
> I don't know about you, but (running away from such things like "I feel in
> my guts", "I believe that", etc) from the technical point of view and the
> schedule point of view, I think that what matters right now are the
> following points :

Yes the gut is important and I say this sometimes too, but we have
force each other to figure out where that gut (subconscious) is
getting it's reasoning from.  Not so easy.

> - we need to get the AP working fully, following the specifications and
> constraints
> - we need to get a 2.0-M1 out to be able to move to the next step
> - we need to implement a Transaction Manager in 2.0-M1 in order to implement
> a better strategy (ie #2 or #3)
> Really the problem now, and this is what I was trying to workaround with
> option #2, is the atomicity constraint. We need a transaction manager badly,
> and we need it for 2.0

Glad we're converging here.

> , no matter what, before we can implement either #2 or
> #3. And once we get this feature added in the server, then I do think that
> #3 is the best possible option.

Yes yes yes ! I totally agree.

Some things really cannot progress unless some supporting
infrastructure is laid down. The AP subsystem is exactly suffering
from this. Everything must evolve together. Otherwise we wind up
implementing a lot of forced solutions that are less than optimal with
hacks, work arounds and one offs. Things get more complex and brittle
causing more pain down the line.

Let's switch into a mode of cleaning up the shop after all this heavy
sawing and hammering. Sit down and plan a micro strategy with a clear
mind and clean shop. Then knock this out in a solid and easy way.

The AP thing was getting way too hard. I commend you on your effort,
and you're close but it's almost impossible to do without a local
transaction manager. Definitely impossible to do RIGHT without one.

Alex Karasulu
My Blog ::
Apache Directory Server ::
Apache MINA ::
To set up a meeting with me:

View raw message