directory-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Emmanuel L├ęcharny <elecha...@gmail.com>
Subject [Mavibot] Status
Date Wed, 27 Sep 2017 10:16:32 GMT
Hi guys,


I was working on Mavibot those last days, in order to get it ready by
mid-october. My last mail was back to june, since then, I was busy on
many other things.

So...


1) Transaction support

It has been greatly improved, and it's now working for Browse, add and
Delete. That means it's now completed, which was the main feature that
was missing.

Transactions are cross-B-tree, and many operations can be applied within
a transaction. For the record, if you update a B-tree 100 times within a
single transaction, instead of committing the transaction 100 times,
updates goes 40 times faster. On my machine, storing 100 000 long and
the associated value as a String (ie <104, "104">) takes 36 seconds if I
commit the transaction for each tuple being added (2750 updates/s), and
0.9s if I commit every 100 updates (108 500 updates/s).

I still have to clean up the code, and to add some more tests.

FTR, I blogged about Transactions on
https://blogs.apache.org/directory/entry/mavibot-transaction


2) Free Page management

To make it simpler, I just put aside this aspect while working on
transaction. I still have to get it back in the code, and more
important, have the page-reclaimer working. That will require a bit of work


3) Cache

Currently, the code does not use any cache. I have remove the B-tree
cache atm.

The idea is to have oneglobal cache, based on page offsets. Currently we
use a PageHolder in Nodes, I may replace it by an offset to the page it
refers to, the cache will hld the page or will grab it from disk.


Anyway, I'll work on that aspect soon and will update you with what I
come with.


4) Bulk Loader

This has to be reviewed, accordingly to the previous changes.



All in all, the new code is more compact, as I removed many features
that were not really useful atm, like :

- sub-btree support

- per B-tree cache

- in-memory B-trees support


More to come soon !



-- 
Emmanuel Lecharny

Symas.com
directory.apache.org


Mime
View raw message