directory-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Selcuk AYA <ayasel...@gmail.com>
Subject Re: Txn Work Update(2)
Date Thu, 01 Dec 2011 07:37:49 GMT
On Wed, Nov 30, 2011 at 3:26 PM, Emmanuel Lecharny <elecharny@gmail.com> wrote:
> On 11/30/11 11:48 AM, Selcuk AYA wrote:
>>
>> Hi all,
>
>
> hi !
>
>> time to give an update on the txn status. I just checked in some code
>> with explanation of the changes for each project separately. Here is
>> just an overall summary and todo:
>>
>> *I added txn being/commit/abort to ldap requests and ran some test
>> with txns enabled. Put in various fixes to the txn layer based on the
>> tests. So first signs life signs for txns. More tests will be done but
>> I need to digress into AVL first. I also need to handle paged search.
>
> Great !
>
> Paged search is just a a normal search, but one that last longer. The way we
> handle it is by using a counter of sent responses.
>
> Basically, entries are sent one by one to the client, using a
> SearchResultEntry to hold the entry. With the pagedSearch, once we have
> reached the limit set by the client, we stop the sending of entries, and
> hold the cursor in the user's session, so that he can send back a new search
> request for then next entries - if needed -. Is there anything special you
> need to deal with here ? (btw, the cursor will be deleted as soon as the
> session will be closed. I hope this cleanup is effective...)
>
There is not much todo here as you guessed. The only problem is the
transaction context is stored in a thread local variable. When paged
search send partial data, we should store the context somewhere and
hand it off to the thread that will continue processing the search.

>
>>
>> *I have also been working on removing interfaces related to
>> modification from the partition
>
> Hmmm, can you be a bit more specific ?

When I say modification or change, I mean
modify/move/rename/moveAndRename/add/delete. Instead of partitions
implementing these separately, there is a layer above partitions which
handle these operations transactionally. The class that handles this
is called OperationExecutionManager. Partitions expose master table
and indices for the operation execution manager execute the
operations.

>
>
>> and have been trying to get rid of the
>> Store interface. Got rid of the store interface except for config
>> methods and calls from the ui layer. I removed the modifications up
>> but still have to call the modification methods of the partition
>> interface because SchemaPartition triggers some changes based on the
>> logical  change. I believe those changes can be removed to schema
>> interceptor?
>
> We can't modify a schema. We can only add or delete a schema entry. This is
> a limitation, and I think it's mandated by the spec (don't remember where it
> comes from, but the idea is that modifying a schema element might imply a
> global modification on all the entries, something we don't want to accept).
>
>
>> In general, if a partition wants to do a change based on
>> a data update, then it should be able to do it during master table
>> update or by registering an interceptor(For example I changed multi
>> file ldif partition to do file updates when master table is updated).
>> Then we can get rid of the modification methods on partitions.
>
> you mean, the modify/move/rename/moveAndRename methods ?

As mentioned above, by change or modification, I mean any operation
that ends up in update of data. For schema partitions, I believe
changes triggered after logical ldap change operations can be moved to
an interceptor.

>
>>
>> Note that, how search is done is still upto the partiton.
>> DefaultSearchEngine is changed to work with any partition and returns
>> transactioanally consistent data.( it used to work with Store
>> interface. Now it works with Partition interface. ). However, a
>> partition can use another search engine and can use the data in txn
>> logs to provide consistent data.
>
>
> Ok.
>
>>
>> * Would be good to get triggers working as they are a nice way to
>> demonstrate txns. What is the reason trigger tests are ignored?
>
> Because since we have modified the server to not use JNDI anymore, it
> impacted the trigger code (which is still based on JNDI). We don't have time
> now to fix this feature, and we have to review the full code in this area,
> as the guy who contributed it is now gone and missing. Let's say it was a
> choice : we know that it's a cool feature, but it's not an urgent one atm...
>
>>
>> * Would be good to have statement snapshot isolation for
>> EntryFilteringCursor. Not having this doesnt affect ldap requests but
>> might affect code that works with EntryFilteringCursor directly.
>> Currently there are few cases of this especially during startup.
>
>
> I'll buy some more explanation : what is a 'statement snapshot' ?

Say you do a search and get an entry filtering cursor and then you
modify the entries that you get from the cursor. Something like this:

beginTransaction()
cursor = ..
while ( cursor.next() )
{
  entry = cursor.get();
  modify entry.
}

endTransaction()

If we have statement snapshot for the search, cursor will not see any
updates to the master table of index tables done through the
modifications.  This might be useful in some cases.

>
>>
>> * Tried to introduce some rules into how modifications are done: If we
>> think of search engine and data modification layer as a data layer,
>> then under it we have storage level. Storage level might have a cache
>> of entries, or index entries or tuples. When a thread reads an object
>> from the storage layer and wants to update it, another thread might be
>> reading the same object. To avoid confusing the reader thread, object
>> to be changed should be cloned before change is applied.
>
> Absolutely.
>
>> Changed
>> modification layer to follow this rule. As mentioned in the previous
>> emails, default search engine is modifying index entries and this is
>> not good, I  put a workaround for this.
>
> Ok. We need to fix this.
>
> Thanks a lot for the update, Selcuk !
>
>
> --
> Regards,
> Cordialement,
> Emmanuel Lécharny
> www.iktek.com
>

Mime
View raw message