Mailing-List: contact dev-help@directory.apache.org; run by ezmlm
Precedence: bulk
Reply-To: "Apache Directory Developers List" <dev@directory.apache.org>
Received-SPF: pass (nike.apache.org: domain of elecharny@gmail.com designates
 74.125.82.50 as permitted sender)
Message-ID: <529C8403.7060107@gmail.com>
Date: Mon, 02 Dec 2013 13:58:43 +0100
From: =?UTF-8?B?RW1tYW51ZWwgTMOpY2hhcm55?= <elecharny@gmail.com>
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.7;
 rv:24.0) Gecko/20100101 Thunderbird/24.1.1
MIME-Version: 1.0
To: Apache Directory Developers List <dev@directory.apache.org>
Subject: [Mavibot Partition] Cache experiments
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

Hi,

now that the Mavibt partition is working, I was able to profile the
server on a search operation. I do a test where I inject 10 000 entries,
and do 400 000 random searches accross those 10 000 entries.

At first, I realized that the EntryCache was initialized to accept only
1 - yes, *one* - entry. You bet that we get a lot of cache misses !
Fixing that lead to a 10% spedup improvement. The time we spare is the
cost of finding an entry browsing a BTree, even if all the BTree pages
are cached (whch s not likely to be a good idea : better favor a huge
entry cache than a huge page cache, the benefit is immediate for a lower
memory cost...)

Second run : the alias cache was added, and this time, I got some
improvement (the last time I added the alias cache was due to fail,
because, again, the number aliases stored in this cache was configured
to ... 1 !). Another 10% win.

Third run : now, I'm caching the ParentIdAndRdn elements, as we need to
access them to reconstruct the DN for each entry. Obviously, the parent
RDN will always be the same... Gain : another 10%.

I'm now able to do around 15 000 searches per second on my laptop,
against around 11 000 before (note : this is a test done with no network
involved).


Not bad...

Althouh the cache configuation now requires a lot of love in order to be
used for real, but this is not something big to do...

More to come after the next profiling session !

-- 
Regards,
Cordialement,
Emmanuel Lécharny
www.iktek.com