On Wed, May 28, 2008 at 9:06 PM, Emmanuel Lecharny <elecharny@apache.org> wrote:
Hi guys,


yesturday, I spent some time running my own little test - I still have to create a project to store this test btw - on the bigbang branch, to compare what we get with what we have on trunk.

Basically, without an once of optimization, trunk is 55% faster than bbb (5000 search req/s against 3200 search req/s).

So where do we eat too much CPU in bbb ?

The big difference is in method :
org.apache.directory.server.core.jndi.ServerDirContext.search(Name, ExprNode, SearchControls)

For 10 000 searches, it costs 433 seconds in bbb and 218 seconds in trunk (so it's two time slower). Not a surprise :) What is interesting is that in bbb, this method spend one third of its time (120secs) calling this method :
org.apache.directory.server.core.jndi.NamingEnumerationAdapter.<init>(EntryFilteringCursor)
(this method does not exist in trunk).

Down the stack, after having passed through all the interceptors, the search() method doing the entries retrieval from the backend cost _exactly_ the same in bbb : 39 seconds in bbb, 40 seconds in trunk. Considering that this part has not been optimized at all, this is quite a good news.

So what's the biggest difference ?

We have this method call in bbb :
org.apache.directory.server.core.filtering.BaseEntryFilteringCursor.next()

that is not present in trunk. It consumes 211 seconds. This is where we are filtering all the entries. The bad thing is that entries are cloned at least three times, and this is a VERY costly operation. It's easy to improve.

Base line is that I'm really confident that we will quickly get some great improvement in bbb, and I'm also pretty confident that we will be able to make bbb faster than trunk.

More to come later this week !

Thanks Emmanuel for this report.  I'm eager to start looking into how we can fix some of these problems you've found.

Alex