directory-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Selcuk AYA <ayasel...@gmail.com>
Subject Re: txn conflict handling and tests
Date Mon, 09 Apr 2012 17:51:47 GMT
On Mon, Apr 9, 2012 at 12:18 AM, Emmanuel Lécharny <elecharny@gmail.com> wrote:
> Le 4/8/12 8:55 PM, Selcuk AYA a écrit :
>>
>> Hi,
>
> Hi !
>
>> I have finally checked in the tests fixes for handling txn conflicts.
>> I added some test and they are passing but I am having some problems
>> with the test framework. First a summary of how txn conclicts are
>> handled and why the tests are kind of important.
>>
>> Txn conclicts detection and handling deals with cases which  would
>> normally be handled by locks. When a RW txn tries to commit and it
>> detects a conflict, it aborts and retries. In order to be able to
>> retry, each operation context saves itself before being executed and
>> is reset based on the saved information if txn aborts because of a
>> conflict and needs to retry. For example and add operation context
>> saves the original entry it wants to add and a modification context
>> saves the original list of modifications it needs to apply.
>
> FYI, the original entry is always read and stored in the OperationContext
> when the operation is received. This is done by the eagerlyPopulateFields()
> method in DefaultOperationManager. If the OperationContext is kept in memory
> until the operation is fulflled (even if you have to rollback it because a
> RW txn has failed), then you have the original entry available.

am aware of that. Original entry has to be reread in eagerlyPopulate
fields when txn is retried because txn has a different start time when
it is retried and it will see a different world.
>
>
>> So the
>> overall flow for a thread doing RW txn is :
>>
>> retry:
>> begintxn
>>
>> if (retry after conflict)
>>    restore context from saved information
>>
>> save context
>>
>> execute
>>
>> commit
>>
>> if (conflict)
>> {
>>  abort
>> goto retry
>> }
>>
>> So when we have two threads adds a telephone number to the same entry,
>> conflict handling makes sure they are serialized and the resulting
>> entry has two telephone numbers.
>
> Sounds good. The only thing we have to take care of here, is that we may
> have to go through the full chan in order to update the entry with the
> potentially modified CollectiveAttributes (not sure though).
>
when txn is rtried, we go through the whole chain again. Retrying
means doing everything for the operation from scratch.

>
>>
>>
>> There are a couple of tests which verifies scenarios similar to above
>> are executed correctly. However, these are multithreaded tests and we
>> have potential problems because of the ChangeLog. Say I have two
>> threads(say T1) one of which deletes dn (cn=test1, ou=system) while
>> another one tries to modify it concurrently(say T2). An execution like
>> this is fine from the txns point of view:
>>
>> 1)T2 starts txn
>> 2)T1 starts txn
>> 3)T1 executes
>> 4)T1 commits
>> 5)T2 executes
>> 6)T2 commits and gets conflict exception
>> 7)T2 retries and gets a nosuchentry exception.
>>
>> In the above scenario, at step 5, changelog already logs the logical
>> operation modify. When revert operation is done at the end of the
>> test, reverting the modify operation gets an exception. Also change
>> log logs two operations at step 3 and 5, however the actual ordering
>> of these two operations can be different than the ordering of the
>> logged entries. For now I ignored the exceptions thrown during the
>> revert operation and limited the number of threads during conflict
>> testing to two but going forward this seems problematic. I am not sure
>> how to work around this because changelog seems essential for the
>> current testing scheme. Any suggestions are welcome.
>
> Just disable the changeLog then. This is not a mandatory element, i has been
> used to ease the tests and improve the server performance when running many
> tests ; we then don't have to restart the server and reinject all the data,
> saving us tons of time !
>
> In the annotations, just add the 'enableChnageLog=false :
>
>    @CreateDS(enableChangeLog = false, name = "PasswordPolicyTest")
>
> You can check the PasswordPolicyTest class to see how it's used.

Got it. Will take a look.
>
> Thanks !
>
>
> --
> Regards,
> Cordialement,
> Emmanuel Lécharny
> www.iktek.com
>

Mime
View raw message