directory-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Emmanuel Lecharny <elecha...@apache.org>
Subject Re: [Schema Refactoring] Serious side effect...
Date Mon, 23 Nov 2009 23:19:44 GMT
More inline...

Alex Karasulu wrote:
> Hi Emmanuel,
>
> On Mon, Nov 23, 2009 at 3:31 AM, Emmanuel Lecharny <elecharny@apache.org>wrote:
>
>   
>> Hi,
>>
>> when we modify the schema, for instance by adding a new AT, in order not to
>> break the current schema, we apply the modification to a cloned Registries.
>> So far, so good. And if the new registries is ok with the newly added
>> SchemaObject, we swap the registries.
>>
>> Now, this create a dull issue : all the ServerEntry instances still point
>> to the previous registries, and as we clean the previous registries to avoid
>> memory leaks, all those instances now point to non existing SchemaObjects
>> (more specifically, each instance has attributes which contain a reference
>> to the AttributeType schemaObject they are associated with) !
>>
>>
>>     
> We cannot hold references to AttributeType or other schema objects anymore.
> Instead we need to perform a lookup everytime to get these schema objects
> from the registries via the schema manager.  This way when the swap occurs
> the new objects are extracted every time.  No member references should be
> made to schema objects any longer.
>
> Question : how can we avoid this problem ?
>
>
> See above.
>   
We discussed about this point this afternoon with Pierre Arnaud, but I 
didn't decide anything before having updated the wiki with a page 
describing the current data structure and all the interactions between 
objects. The page is 
http://directory.apache.org/apacheds/1.5/apache-ds-schemamanager.html

The big problem with this approach (which was what was implemented 
before) is that each lookup is costly, and we do a lot of them while 
processing a request.

Having the AT stored into the AttributeEntry solve this problem, as we 
have a direct access to the data : we just do a single lookup at the 
beginning.
>
>   
>> My idea is to apply the modification to the initial registries once it has
>> been proven that the registries alteration leave the Regsitries consistent.
>> We will just clean the cloned registries, and don't swap anymore.
>>
>>
>>     
>
> I think this is not a good idea.  The swap is a clean approach that does the
> job just fine.  Otherwise if there are sticking points in the registries we
> will have problems.  This is too much to do to solve this issue.  Instead we
> need dynamic look ups of all the schema objects or we need a update pattern
> to inform of schema changes to swap out these objects.
>
> I am thinking of the Observer or Listener notification patterns here.
> Otherwise we will have to use dynamic lookups every time which can cost us a
> lot.
>   

We discussed this point with Alex extensively lately. This Observer/Listener pattern sounds
good, but IMO, it does not solve the problem. 
1) We have to deal with all the serverEntries being currently processed, and maybe being modified.
A ServerEntry is not an atomic object, and we don't want to deal with the extra complexity
of a event occuring while we are in the middle of a modification of this ServerEntry
2) Many ServerEntries might be stuck in memory, waiting for a thread to be free. This is the
case for every partial ServerEntry waiting for some more bytes from the client. 
3) We have many places where we store cached ServerEntries. The question is how do we update
them ?
4) What do we do with the Original entry, which is stored into each OpContext, as we may need
it later ? Do we update it too ? IMO, that would defeat the purpose of this object
5) ServerEntries are serialized in the Backend, and if we modify the schema, it will most
certainly impact them. If they are not migrated, they might not be usable anymore after the
schema modification. Also if we have millions of entries, changing them online is probably
not realistic. Anyway, the admin has to deal with this problem in any case

I don't see how possibly we can deal with a schema modification live, except for a few modifications
:
- AT, OC, S, MR, C, N and SC, and only for Add or Move operations
- schema enabling

Any other operation (delete, modify, rename, disabling a schema) are most certainly leading
to dire errors, something an administrator will not want to experiment in production. IMO,
they should be forbidden on a working base. Such operation is like manipulating a loaded weapon
with no safety...

>
>   
>> Does it sound good to you, or do you have any better idea ?
>>
>>
>>     
> No I think we either go with expensive dynamic lookups all over the place or
> we utilize a update notification mechanism via the Observer or Listener
> patterns.
>
> Does this sound like a viable alternative?
>   
For the reason I mentionned, I don't think that any alternative is ok. 
We probably don't have a perfect solution because there are none. As we 
say : "any problem vanishes when there is no solution"...

More seriously, I don't think we need a dynamic schemaManager for a LDAP 
server in production : admins don't change such a critical thing in 
production, except those who are insane or desesperate. We must accept 
the idea that we might have a downtime, we just have to minimize it.

Unless someone has a genious idea !
> Regards,
>
>   


-- 
--
cordialement, regards,
Emmanuel L├ęcharny
www.iktek.com
directory.apache.org



Mime
View raw message