directory-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Selcuk AYA <ayasel...@gmail.com>
Subject Re: merging txn branch back into the trunk
Date Tue, 08 May 2012 05:18:26 GMT
On Mon, May 7, 2012 at 3:50 PM, Emmanuel Lécharny <elecharny@gmail.com> wrote:
> Hi,
>
> some few comnts inline...
>
>
> Le 5/7/12 11:04 AM, Selcuk AYA a écrit :
>>
>> Hi All,
>>
>> ****CHANGES AFFECTING THE CURRENT  CODE*********
>>
>> *** The general motive is to implement a logical transaction layer
>> above partitions. This transaction layer keeps a log of logical
>> changes. It works above partitions and expects partitions to  conform
>> to a certain data model and give some consistency guarantees:
>>
>>                        1) Partitions should expose a MasterTable which
>> stores Entry objects. They also should expose the expected system
>> indices and can expose user indices. Master and Index tables are
>> basically (key, value) tables.
>>                        2) Supported modification operations on Master
>> and Index tables should be atomic
>
> RdnIndex may be updated more than once when adding an entry, do to the
> 1/SubLevel indexes. If this break atomicity, then we have to deal with that.
> (see below)

multiple updtes should not be an issue.
>
>>
>>
>> ***Another goal in this effort was to move the operation execution
>> logic above partitions so that transactions could be implemented
>> independent of partitions with the above consistency guarantees.
>> Towards this goal, DefaultOperationExecutionManager has been
>> introduced. This mostly copies the logic in AbstractBTreePartition and
>> executes updates on master tables and indices using log edits rather
>> than direct updates on them. These log edits are then applied to
>> partitions using the atomic modifications on Master and Index tables
>> exposed by partitions. This class is where we really need to review
>> well to ensure things are OK.
>
> Ok, will review this.
>
>> ******MERGING********
>>
>> - One and Sub level index removal will probably impact
>> DefaultOperationExecutionManager.
>
> The question is to what extent these changes will impact the code.
>
>> - From emails, I understand the Index interface assuming UUID might be
>> a problem with the recent changes. Maybe this needs to be changed too.
>
> Hmm, not sure. Moving to get rid of any other type to use UUID instead is
> the right choice. All the indexes contains relation between a key and some
> Entry's UUIDs (forward index), and between an UUID and some keys (reverse
> index). The reverse index is used by the search engine.
>
> Do you see any issue with that ?

What you described is what is done in the branch right now. I thought
you were going to store nbdescendants and other auxiliary information
along with UUID. That is why I thought the change would be necessary.
How is this auxiliary information stored now?

>
>>
>>
>> Either before or both before and after the merge, we should run a test
>> with concurrent threads(read+write) and clear out any remaining
>> issues.
>
> Ok.
>
>>
>>
>> *******TODO********
>>
>> After the merge, I will work on implementing crash recovery part which
>> will complete the transactional layer changes.
>
> Fine !
>
> Great job !
>
>
>
>
> --
> Regards,
> Cordialement,
> Emmanuel Lécharny
> www.iktek.com
>

Mime
View raw message