directory-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Emmanuel Lécharny <>
Subject Re: txn conflict handling and tests
Date Mon, 09 Apr 2012 07:18:56 GMT
Le 4/8/12 8:55 PM, Selcuk AYA a écrit :
> Hi,
Hi !
> I have finally checked in the tests fixes for handling txn conflicts.
> I added some test and they are passing but I am having some problems
> with the test framework. First a summary of how txn conclicts are
> handled and why the tests are kind of important.
> Txn conclicts detection and handling deals with cases which  would
> normally be handled by locks. When a RW txn tries to commit and it
> detects a conflict, it aborts and retries. In order to be able to
> retry, each operation context saves itself before being executed and
> is reset based on the saved information if txn aborts because of a
> conflict and needs to retry. For example and add operation context
> saves the original entry it wants to add and a modification context
> saves the original list of modifications it needs to apply.
FYI, the original entry is always read and stored in the 
OperationContext when the operation is received. This is done by the 
eagerlyPopulateFields() method in DefaultOperationManager. If the 
OperationContext is kept in memory until the operation is fulflled (even 
if you have to rollback it because a RW txn has failed), then you have 
the original entry available.

> So the
> overall flow for a thread doing RW txn is :
> retry:
> begintxn
> if (retry after conflict)
>     restore context from saved information
> save context
> execute
> commit
> if (conflict)
> {
>   abort
> goto retry
> }
> So when we have two threads adds a telephone number to the same entry,
> conflict handling makes sure they are serialized and the resulting
> entry has two telephone numbers.
Sounds good. The only thing we have to take care of here, is that we may 
have to go through the full chan in order to update the entry with the 
potentially modified CollectiveAttributes (not sure though).

> There are a couple of tests which verifies scenarios similar to above
> are executed correctly. However, these are multithreaded tests and we
> have potential problems because of the ChangeLog. Say I have two
> threads(say T1) one of which deletes dn (cn=test1, ou=system) while
> another one tries to modify it concurrently(say T2). An execution like
> this is fine from the txns point of view:
> 1)T2 starts txn
> 2)T1 starts txn
> 3)T1 executes
> 4)T1 commits
> 5)T2 executes
> 6)T2 commits and gets conflict exception
> 7)T2 retries and gets a nosuchentry exception.
> In the above scenario, at step 5, changelog already logs the logical
> operation modify. When revert operation is done at the end of the
> test, reverting the modify operation gets an exception. Also change
> log logs two operations at step 3 and 5, however the actual ordering
> of these two operations can be different than the ordering of the
> logged entries. For now I ignored the exceptions thrown during the
> revert operation and limited the number of threads during conflict
> testing to two but going forward this seems problematic. I am not sure
> how to work around this because changelog seems essential for the
> current testing scheme. Any suggestions are welcome.
Just disable the changeLog then. This is not a mandatory element, i has 
been used to ease the tests and improve the server performance when 
running many tests ; we then don't have to restart the server and 
reinject all the data, saving us tons of time !

In the annotations, just add the 'enableChnageLog=false :

     @CreateDS(enableChangeLog = false, name = "PasswordPolicyTest")

You can check the PasswordPolicyTest class to see how it's used.

Thanks !

Emmanuel Lécharny

View raw message