directory-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Selcuk AYA <ayasel...@gmail.com>
Subject Re: We may need to add a hasChildren() method in the Partition interface
Date Fri, 02 Dec 2011 10:49:00 GMT
On Fri, Dec 2, 2011 at 12:40 PM, Emmanuel Lécharny <elecharny@apache.org> wrote:
> On 12/2/11 11:03 AM, Selcuk AYA wrote:
>>
>> Hi,
>> with a little digression, here are some general ideas on the schema
>> changes:
>>
>> I think we should handle schema changes in a way something like this:
>> 1)quiesce all ldap operations on all partitions.
>
>
> Sure. But we can even be a bit more strict : schema deletion could also be
> forbidden (this is what AD does, btw). In fact, we could even just disable a
> schema element, but not delete it, so that entries depending on a deleted
> schem element is still valid.
> OTOH, changing the schema live is a dangerous operation, most of the case
> the server is usually shutdown, and restarted when the schema is up to date
> (which doe snot solve the case of entries pointing to removed schema
> elements, btw).
>
> We may want to add a kind of barrier which is raised when a schema
> modification starts (ie, all the new operation are enqueued, waiting for the
> schema modification is done), and the schema modification is waiting for all
> the pending operation to be completed (and that include the paged search).
> But I don't think it's mandatory : the schema is used through the
> registries, which is always consistant (see down this mail)
>
>> 2)change schema, and maybe update registries.
>
> We first update the registries, to check if its consistant or not. If it is,
> then we update the LDIF partition, and we then switch the updated registries
> with the current registries.
>
> So in fact, the steps are :
> 1) clone the registries
> 2) update the cloned registries
> 3) check if it's consistant
> 4) if so, update the schema on disk
> 5) then switch the registries (it's a protected section, the registries
> variable is volatile)
>
> and we are done.

Makes sense.

>
>> 3) Check for inconsistency but if anything goes wrong after updating
>> registries, reinit the registries from on disk schema partition.
>
> we just don't switch the registries in this case : the cloned registries is
> simply ditched
>
>> Reasoning:
>>
>> *Almost every code has read dependency on the schema registries, so
>> when a thread changes the schema and the registries, every operation
>> will be affected by the change. So I do not think letting any other
>> operation to be concurrent with schema change makes sense.
>
> right.
>
>>
>> *At step 3 above, schema entry having a child is only one case of
>> failure. Currently we also seem to clone the registries before doing
>> any change to them. Ignoring all the current problems because of lack
>> of isolation of schema change, lets say we do the following:
>> a)check if hasChildren is false
>> b) clone registries, update them and check for inconsistency
>> c) if everything is OK, update registries
>> d) Update data on disk.
>>
>> If something went wrong during step d and operation failed, cloning
>> the registries and doing the update on the cloned registries wouldnt
>> save us and reinit of registries would be necessary.
>
> the main issue is o-how do we deal with a failure in the middle of the
> modification of the schema on disk ? We don't have any way to restore the
> original schema. We may need to copy the whole schema on disk (a backup)
> before modifying it, so that if we have a failure, we can safely restart the
> server from the backup ( which will not be a backup, btw, but the *real*
> schema, until the check is done, where we can safely discard the 'backup'
> and use the new schema.

My point was actually we might need to reinit the registries after
failure on write to partition but your reply above addresses this. In
txns branch, writing the ldif data to a shadow file and doing a move
handles the atomicity of update on schema entries. This, in effect, is
similar to what you suggest.

>
>>
>> P.S. With txns, hasChildren is not necessary because it is implemented
>> above the partitions using the indices   exposed by partitions.
>
>
> Okie.
>
>
>
> --
> Regards,
> Cordialement,
> Emmanuel Lécharny
> www.iktek.com
>

Mime
View raw message