Mailing-List: contact dev-help@directory.apache.org; run by ezmlm
Precedence: bulk
Reply-To: "Apache Directory Developers List" <dev@directory.apache.org>
Received-SPF: pass (athena.apache.org: domain of elecharny@gmail.com
 designates 74.125.83.50 as permitted sender)
Message-ID: <4EE9E7B1.4090705@gmail.com>
Date: Thu, 15 Dec 2011 13:27:29 +0100
From: Emmanuel Lecharny <elecharny@gmail.com>
Reply-To: elecharny@apache.org
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.6;
 rv:8.0) Gecko/20111105 Thunderbird/8.0
MIME-Version: 1.0
To: Apache Directory Developers List <dev@directory.apache.org>
Subject: Re: Txn Work Update(2)
References: 
 <CALhXAG0+vHAA0qcVMwmX7vDZ-oi=ddGSU-e3Ngixv=j4kuwVLQ@mail.gmail.com>
 <4ED62F11.1090602@gmail.com>
 <CALhXAG3i27D-8p7ZsDxfhr2gdx-hgNgjwqrWyrBRVQ6E7DbSTw@mail.gmail.com>
 <4ED73F10.4050705@gmail.com>
 <CALhXAG2moN7rM4xZzwMiiQ_b1Dsa3_gLQ5wN9JSrGvp=Vp7m4Q@mail.gmail.com>
In-Reply-To: 
 <CALhXAG2moN7rM4xZzwMiiQ_b1Dsa3_gLQ5wN9JSrGvp=Vp7m4Q@mail.gmail.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 8bit

On 12/15/11 12:25 PM, Selcuk AYA wrote:
> Hi all,
>
> update on my status:
>
> * There is going to be a big change in my life soon and I will do a
> big move. So I will be MIA for some time( around 2 months) and after
> that I will have less time to contribute but I will try to keep being
> involved in development and discussions.
Ok. I wish this is a move for a better life :)

>
> * I did a recent check in. There are some tests that fail. I will list
> them below. The basic reason that those tests are failing is that they
> do queries and updates of which  some are transactional, some are not
> and this leads to wrong results.
I spent some time yesterday on the failing tests. What I need here is a 
clue about how to make some operation transactionals. If you can update 
us, that would be great.

> And the basic reason they do queries
> and updates in a non transactional way is that I did not put
> transaction begin/commif/abort( ie, txn demarcation) at CoreSession.
> This is the layer we start the execution of  the basic ldap operations
> but I thought there might be clients of CoreSession which want to
> group operations together and execute them as a single txn(If you
> think this is not necessary, please let me know).
I don't think it's anymore necessary. We have refactored all the 
operations last month so that we never have to re-enter the chain twice 
when processing an operation.

>   Also, there is at
> least search, for which the cursor is kept open above CoreSession, So
> for search, the caller has to handle txn commit, abort.Also there
> might be some cases where operations do not use session at all.
Well, we do some lookups directly on the nexus, as we are inside the 
chain already, and I guess we are using the ongping transaction here. If 
it's not the case, then we should probably have a way to make those 
operations part of the ongoing txn (don't know how difficult it is)

>
>
> As a result, I decided to handle each  caller of CoreSession
> operations separately and this increased the number changes that had
> to be done. I handled ldap protocol handlers and change password.
> Right now StockServerSuite is passing except three tests.
> StockCoreSuite and MigratedStockCoreSuite is failing. The reason is
> they use LDAPCoreConnection and this is not transactional yet.
I have some failures in core-integ. Do you have them too ?
>
> For anybody who wants to get involved, here is the list of todo:
> * Figure out why SaslBind and DelegatedAuthIT in stockserver suite is
> failing. We get a KRBException here and I do not know this module well
> enough.
Will check.
>
> *Handle LdapCoreConnection. This is used in applications which embed
> apacheds i guess?
Yes.
> The basic difficulty is that users of this class get
> a cursor and the transaction should be comitted or aborted only when
> this cursor closes . Moreover, when a cursor is open, user of
> ldapcoreconnection can call other ldap operations and it might make
> sense to group these in a transaction. If this is true, then it should
> be the responsibility of the callers of LdapCoreSession to start and
> end txns. If we are not going to let the user do this, we need to
> store a pointer to txn context in the cursor and end the txn when
> cursor is closed taking into account that the txn might have to be
> upgraded to a readwrite txn from a read only txn if the user does
> changess while holding the cursor open.
I don't think we should expose the txns. It should remain an internal 
mechanism. Also I don't mind if a user has an issue because he is doing 
some changes in the server while processing a search : it's his problem, 
not our. It is very clear that LDAP is not transactionnal, and the suer 
should be aware of this fact. (that just means a user can't expect that 
a long search can return entries which have been modified or deleted 
since the search has started. But the user should not get an exception 
just because an entry does not exists anymore when the cursor is moving 
forward).

At least, this is my undertsanding. Alex, do you agree on that ?
>
> * There are some tests which call session operations directly, these
> should be wrapped in txn begin/end.
ok.
>
> *I put some tests to AVL and jdbm partitions to test txns without the
> effect of interceptors and logical caches. These are multithreaded and
> are good to test the robustness of the data change and query layer and
> txn subsystem. Add more tests like this.
Ok.
>
> * Handle logical caches. As we talked handle these with read write
> locks taking cleanup in  case of an aborted txn into account.
>
> Please let me know if anyone wants to attack any of these. I might
> provide more input if needed.

I will spend some time by the end of this year on those things. We need 
to get a branch that builds and passes tests with flying colors, even if 
it's not perfect.

Also I see that there are TODO into the DefaultOperationExecutionManager 
class :

     public void delete( Partition partition, DeleteOperationContext 
deleteContext ) throws LdapException
     {
         Dn dn = deleteContext.getDn();

         //TODO TODO TODO REMOVE THIS
         partition.delete( deleteContext );

What should we do ?


-- 
Regards,
Cordialement,
Emmanuel Lécharny
www.iktek.com