directory-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Emmanuel L├ęcharny <elecha...@symas.com>
Subject Mavibot cache experiment
Date Tue, 10 Sep 2013 15:40:49 GMT
Hi guys,

yestrday and today, I was implementing a cache in Mavibot to replace the
WeakReferences we were using previously. The rational was that with the
WeakReferences, we were unable to inject more than 50 000 entries in the
server (at this point, the GC is just going crazy trying to free some
WeakReferences, and it slows down the server to a pint it's unusable...)

So after a day fighting with an EhCache implementation in Mavibot, I was
able to load 100K entries in the server. So far, so good, except that
the performances are anything but good. I can add 26 entries per second,
and fetch 555 entries per second. Worse than JDBM...

Why is it so slow, especially for a search operation ? There are two
reasons :
- first, I configured the cache to store only 1000 elements (mainly nodes)
- second, when we try to update a leaf, we mostly have to load it from
disk, as we rarely have it in memory
- third, a leaf contains from 8 to 16 entries, and everytime we fetch a
leaf from disk, we have to deserialize the Entries, which is extremely
costly

Fixing this third problem would save us a lot of time, and it's a matter
of adding one level of indirection (the entries would be kept as byte[],
and deserialized only when needed).

If anyone has a better idea...

Mime
View raw message