Return-Path: X-Original-To: apmail-directory-dev-archive@www.apache.org Delivered-To: apmail-directory-dev-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id CB08674DC for ; Tue, 1 Nov 2011 03:36:20 +0000 (UTC) Received: (qmail 46819 invoked by uid 500); 1 Nov 2011 03:36:20 -0000 Delivered-To: apmail-directory-dev-archive@directory.apache.org Received: (qmail 46625 invoked by uid 500); 1 Nov 2011 03:36:19 -0000 Mailing-List: contact dev-help@directory.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: "Apache Directory Developers List" Delivered-To: mailing list dev@directory.apache.org Received: (qmail 46618 invoked by uid 99); 1 Nov 2011 03:36:17 -0000 Received: from nike.apache.org (HELO nike.apache.org) (192.87.106.230) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 01 Nov 2011 03:36:17 +0000 X-ASF-Spam-Status: No, hits=1.5 required=5.0 tests=FREEMAIL_FROM,HTML_MESSAGE,RCVD_IN_DNSWL_LOW,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (nike.apache.org: domain of akarasulu@gmail.com designates 74.125.82.178 as permitted sender) Received: from [74.125.82.178] (HELO mail-wy0-f178.google.com) (74.125.82.178) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 01 Nov 2011 03:36:09 +0000 Received: by wyh13 with SMTP id 13so472019wyh.37 for ; Mon, 31 Oct 2011 20:35:49 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=mime-version:sender:in-reply-to:references:date :x-google-sender-auth:message-id:subject:from:to:content-type; bh=pkIhd+CaUCVD7o/sCmlWWbwFMIVe9+pICzsTcrxOOtw=; b=BxzMTh9goTuMSMCVyGLvFdXLqJhuqRmatwMC38qZZucErO5Q9Vwt/K00816fhfS0g6 XNXIet1jmcs69fwHbSqSWoAiL/m0kBs9Es7R8Z56QSRkCtzt1wG/u29/iPW62kTkCh1z PKD18vaZBeLO9mhVXdCzsy2ohKA9zMdO13MjY= MIME-Version: 1.0 Received: by 10.227.205.76 with SMTP id fp12mr23773262wbb.17.1320118549458; Mon, 31 Oct 2011 20:35:49 -0700 (PDT) Sender: akarasulu@gmail.com Received: by 10.180.8.232 with HTTP; Mon, 31 Oct 2011 20:35:49 -0700 (PDT) In-Reply-To: References: Date: Tue, 1 Nov 2011 05:35:49 +0200 X-Google-Sender-Auth: 1nP8minRxCW4xJdRaPb2yv_X-Fo Message-ID: Subject: Re: TXN work update From: Alex Karasulu To: Apache Directory Developers List Content-Type: multipart/alternative; boundary=001517592dfa33af3e04b0a40da8 X-Virus-Checked: Checked by ClamAV on apache.org --001517592dfa33af3e04b0a40da8 Content-Type: text/plain; charset=ISO-8859-1 On Thu, Oct 20, 2011 at 4:24 PM, Selcuk AYA wrote: Excuse the late response. Better late than never right ... SNIP ... > *Master table, index and index cursor wrappers: These provide > transactionally consistent views of entries and index data. They merge > what they read from their wrapped objects with the txn log manager. To > make this easier, txn manager keeps track of index changes for a > read/write txn in a summary form. One thing to note is that not all > information that wrappers expose is transactionall consistent. One > example is count methods. No need to %100 accurate on this. For these > cases, the wrapper just calls the wrapped object. > > Makes perfect sense to not bother with the count methods: slight variations will not at all impact the optimizer's results. > *For all the above, interfaces are defined in core-api and > implementations and internal interfaces are in core. Also moved, > index, indexcursor, table and related interfaces and helper classes to > core-api from xdbm-partition. > > I guess the xdbm-partition Maven module is now going to be pretty damn thin or non-existent. Was this module destroyed and if not what actually remains? > The todo is the following: > *add changes to keep track of dn space changes. > *test txn manager services > *move the modification code path in AbstractBTreePartition in xdbm to > high up to core. Probably modifications should be done in > partitionNexus and partition nexus should handle preparing txn log > edits and wal them. The overall flow for modifications will be: > -DefaultDirectoryService: > -begintxn > -execute interceptor chain > - handle txn abort, or conflict. > > Moving this into the PartitionNexus might not be a good idea but no problem for now we can move it later. Let me explain why: Eventually we're going to enable a root Partition with Partition nesting and so when this happens the PartitionNexus will just be another nestable Partition since these will have to handle routing based on DN to other partitions residing/nested under it. I see two possible locations for this functionality: (1) Let the InterceptorChain itself handle this since it can demarcate the start and end of calls into the chain with Txn begin and abort/commit calls. It does this by making calls against the TxnManager which I guess is the entire facade for the transaction subsystem. ---OR--- (2) Handle Txn demarcation within the CoreSession. However this might not be optimal due to the need to handle additional logic which might be required for handling chain re-entry concerns. NOTE: I've not actually looked at the code after these major moves so my advice might not be very dependable. I will try to setup my environment to get a better idea of these matters. However for the time being do whatever actually makes this thing work. Let's follow an agile methodology. This thing is big. So let's get it working with solid test coverage then we can actually look at shuffling things around to optimal positions. Not saying what you've chosen is wrong ... it might just present the need for some additional refactoring when other features might need to be introduced. These are some of the biggest changes to the architecture to have taken place in years and you're doing a great job. *move xdbm-search to core as well. Making search transactional will > mostly be mostly mechanica after this point I think(hope). It should > just use the wrappers the txn manager provides for index, master and > cursors it gets from the partitions. > > *handle caches various interceptors keep. I am thinking of handling > this with a common read-write lock. > > This was the latest issue for which I see some more threads. Will look at that as well. Again sorry about late responses. Regards, Alex > regards > Selcuk > > On Thu, Oct 20, 2011 at 4:05 PM, Selcuk AYA wrote: > > Hi all, > > > > Emmanuel wants to update the txns branch and I think it is a good time > > to give a good status update. Following is what I have been doing and > > the plan: > > > > *Log Manager: A general log manager that can be used by txns as well. > > Note that these logs are on disk. This is tested. > > > > *Txn Manager:Most of the services that should be provided by the txn > > manager is coded: > > begin, commit, abort, keeping > > > -- Best Regards, -- Alex --001517592dfa33af3e04b0a40da8 Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable
On Thu, Oct 20, 2011 at 4:24 PM, Selcuk AYA <ayaselcuk@gmail.co= m> wrote:

Excuse the late response. Be= tter late than never right ...

SNIP ...
=A0
*Master table, index and index cursor wrappers: These provide
transactionally consistent views of entries and index data. They merge
what they read from their wrapped objects with the txn log manager. To
make this easier, txn manager keeps track of index changes for a
read/write txn in a summary form. One thing to note is that not all
information that wrappers expose is transactionall consistent. One
example is count methods. No need to %100 accurate on this. For these
cases, the wrapper just calls the wrapped object.


Makes perfect sense to not bother with= the count methods: slight variations will not at all impact the optimizer&= #39;s results.
=A0
*For all the above, interfaces are defined in core-api and
implementations and internal interfaces are in core. Also moved,
index, indexcursor, table and related interfaces and helper classes to
core-api from xdbm-partition.


I guess the xdbm-partition Maven modul= e is now going to be pretty damn thin or non-existent. Was this module dest= royed and if not what actually remains?
=A0
The todo is the following:
*add changes to keep track of dn space changes.
*test txn manager services
*move the modification code path in AbstractBTreePartition in xdbm to
high up to core. Probably modifications should be done in
partitionNexus and partition nexus should handle preparing txn log
edits and wal them. The overall flow for modifications will be:
=A0 -DefaultDirectoryService:
=A0 =A0 =A0 =A0 =A0-begintxn
=A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0-execute intercepto= r chain
=A0 =A0 =A0 =A0 =A0- handle txn abort, or conflict.


Moving this into the PartitionNexus mi= ght not be a good idea but no problem for now we can move it later. Let me = explain why:

Eventually we're going to enable = a root Partition with Partition nesting and so when this happens the Partit= ionNexus will just be another nestable Partition since these will have to h= andle routing based on DN to other partitions residing/nested under it.=A0<= /div>

I see two possible locations for this functionality:

(1) Let the InterceptorChain itself handle this sinc= e it can demarcate the start and end of calls into the chain with Txn begin= and abort/commit calls. It does this by making calls against the TxnManage= r which I guess is the entire facade for the transaction subsystem.

---OR---

(2) Handle Txn demarc= ation within the CoreSession. However this might not be optimal due to the = need to handle additional logic which might be required for handling chain = re-entry concerns.=A0

NOTE: I've not actually looked at the code after th= ese major moves so my advice might not be very dependable. I will try to se= tup my environment to get a better idea of these matters.

However for the time being do whatever actually makes this thing= work. Let's follow an agile methodology. This thing is big. So let'= ;s get it working with solid test coverage then we can actually look at shu= ffling things around to optimal positions. Not saying what you've chose= n is wrong ... it might just present the need for some additional refactori= ng when other features might need to be introduced.=A0

These are some of the biggest changes to the architectu= re to have taken place in years and you're doing a great job.

*move xdbm-search to core as well. Making search transactional will
mostly be mostly mechanica after this point I think(hope). It should
just use the wrappers the txn manager provides for index, master and
cursors it gets from the partitions.

*handle caches various interceptors keep. I am thinking of handling
this with a common read-write lock.


This was the latest issue for which I = see some more threads. Will look at that as well.

= Again sorry about late responses.

Regards,
Alex=A0
=A0
regards
Selcuk

On Thu, Oct 20, 2011 at 4:05 PM, Selcuk AYA <ayaselcuk@gmail.com> wrote:
> Hi all,
>
> Emmanuel wants to update the txns branch and I think it is a good time=
> to give a good status update. Following is what I have been doing and<= br> > the plan:
>
> *Log Manager: A general log manager that can be used by txns as well.<= br> > Note that these logs are on disk. This is tested.
>
> *Txn Manager:Most of the services that should be provided by the txn > manager is coded:
> begin, commit, abort, keeping
>



--
=
Best Regards,
-- Alex

--001517592dfa33af3e04b0a40da8--