directory-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Emmanuel L├ęcharny <>
Subject [Mavibot] file layout changes
Date Fri, 04 Oct 2013 08:06:26 GMT
Hi guys,

I'm curently modifying the file layout for the managed BTree. There are
many reasons I'm doing t, let me expose them :

1) first of all, there is now two differnet kind of BTree : in-memory
and Managed. I tried really hard to keep one single class to handle the
two kind of btrees, but it was really too hard with generics playing bad

2) the tests I conducted before going to SF 2 weeks ago shown that we
were able to load more than 50 000 entries when using Mavibot managed
btrees, but at a price which is simply unacceptable. The pb was that at
some point, adding an entry means loading a Leaf from disk,
deserialising all its keys and values, just to add a new key and value,
then the page get quickly discarded from the cache. With 12 elements per
Leaf, we are doing 11 useless deserialisation.

Now, I'm defining some KeyHolder and ValueHolder to keep either the
deserialized key or value, or the byte[] of their serialized
counterpart. They are deserialized on demand, saving a hell lot of
costly processing.

That requires a modification of the file layout, as I need to store one
more information, ie the length of the serialized data. This is an
on-going work.

Just wanted to keep you informed !

Emmanuel L├ęcharny 

View raw message