directory-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Alex Karasulu <>
Subject Re: Problem of JDBM being hard(impossible) to reconfigure once it is initialized
Date Wed, 23 May 2012 19:05:00 GMT
On Wed, May 23, 2012 at 1:36 AM, Emmanuel Lécharny <> wrote:
> Le 5/23/12 12:01 AM, Selcuk AYA a écrit :
>> I  do not know the OSGI jargon but I believe,  at the end, changing
>> these should reduce to something like this:
>> 1) quiesce the necessary ldap operations: The simplest thing to do
>> would be to block all operations quiesce all the outstanding
>> operations. In some cases, it would be enough to block only searches.
>> There might be some cases where you do not need any blocking at all. I
>> do not
> I'm not comfortable with the idea of quiscing the ldap operations. The
> server is supposed to work 24x7, and if we do add some way to modify the
> backend config on the fly, then the user should not notice that.


I agree with Emmanuel here. Such changes should not interrupt the
operation of the server.

> See my comments below.
>> 2) do the configuration: In the end I am assuming the configuration is
>> delegated to the component itself to do the reconfig.
>> 3) unblock the operations.
>> For index add:
>> * queisce any ldap operation that might modify the data. Then notify
>> the partition to add the index. Partition will scan the master table
>> and build the index and swap its indexed attributes. Then unlock the
>> operations.
> Adding an index should be done while the server is still able to process
> requests. The thing is that the newly added server should not be available
> until it has been created.

I guess you mean the newly added index here above.

The real problem is that we won't be able to
> parse the master table fast enough to have all the subscequent modifications
> into it, so this is a two steps operation :
> - do a full scan search on the master table, and create the additional
> index. We should be safe here, because we will use the current version of
> the master table when we start the search.
> - then for every modification done after the beginning of step 1, apply
> those changes to the index. If we have some more changes since we started
> this step, then do it again.
> Now, we can use this index.

OK let me see if I understand. To add an index we:

1). Note the version (V0) when the request to build the index arrives.
Start a full master table scan at version V0 to build the new index.

2). Once the index has been built for this version acquire an
exclusive write lock to prevent writes.

3a). If no changes occurred while building the index, meaning we're
still at V0. Then we enable the index and release the exclusive write

3b). If changes have occurred while building the index, then we update
the index with changes V1-V0. Then we enable the index and release the
exclusive write lock.

>> For index delete:
>> * block all operations, and remove the index from the indexes list.
>> Maybe commit this change in partition config. unlock changes. Then
>> continue deleting  index file.
> Here, it's simpler. The operation using this index will continue to use it,
> until they are done. New operations won't be allowed to use it. Once the
> number of requests using this index is 0, then the index can be deleted. I
> don't think we need to block any operation here.

We just need an index state. It should be marked as deleted or offline
so when a new search comes in while waiting for the others to finish
using this index we simply do not use the index. So we need some index
state information.

>> For suffixdn change:
>> * I believe this is a rename operation. again block all operations and
>> do the rename. Then unblock.
> changing teh suffix DN is just a matter of modifying the configuration, and
> the DnNode structure. The problem is that we rebuild all the entries' DN
> using the partition suffixDn, so we have to be careful when doing so. The
> request being processed when the rename occurs should be fulfilled with the
> initial suffixDn.


>> Working directory change;
>> * Block all opreations and let the parition change its config dir. For
>> jdbm, this would be a copy of it working directory.
> Here, this is an administrative task. Not sure we want to do that on a
> running server...

What other option is there?

You should not have to stop and re-start the server just to rename a
partition suffix. We simply need to mark (as we did for indices) the
partition as offline waiting until all current in progress operations
complete on that partition. New requests should not be processed for
it. Maybe we can return an unwilling to perform result response.

Then bring the partition online in the new suffix after doing the copy
and making all the needed changes. This should be a really fast
operation because the file system operation is rename. The other
suffix configuration changes are point changes. It should not take
long regardless of the size of the partition.

Best Regards,
-- Alex

View raw message