Mailing-List: contact dev-help@directory.apache.org; run by ezmlm
Precedence: bulk
Reply-To: "Apache Directory Developers List" <dev@directory.apache.org>
Received-SPF: pass (athena.apache.org: domain of elecharny@gmail.com
 designates 72.14.220.154 as permitted sender)
DomainKey-Signature: a=rsa-sha1; c=nofws;
        d=gmail.com; s=gamma;
        h=message-id:date:from:reply-to:user-agent:mime-version:to:subject:content-type:content-transfer-encoding:sender;
        b=RmuznxwU/hnjNmhLajstE4GYmDt6OjjvjxSDsEtqF0JfB37hpepXqXug16pZJmmsU79OPv7OZCYF134P8QTrTUsz7Pv6Y+vAnlE17QhHUziLVFwgG6DCe07W1cluJG05Yx3CEj2cui0hoduCLZWs8Xb3BXA/fgc4eyppvthn0RU=
Message-ID: <481856DE.7060904@apache.org>
Date: Wed, 30 Apr 2008 13:24:14 +0200
From: Emmanuel Lecharny <elecharny@apache.org>
Reply-To: elecharny@apache.org
User-Agent: Thunderbird 1.5.0.14ubu (X11/20080306)
MIME-Version: 1.0
To: Apache Directory Developers List <dev@directory.apache.org>
Subject: [ServerEntry] Next step
Content-Type: text/plain; charset=ISO-8859-1; format=flowed
Content-Transfer-Encoding: 8bit
Sender: Emmanuel Lecharny <elecharny@gmail.com>

Hi guys,

I have finished to modify the code to get the ServerEntry serialized 
into the backend. It was not really easy, but it's done. It also has 
some impact :
- I had to modify JDBM in order to be able to deserialize the entries 
(just added an accessor to a class)

A mail has been sent to Alex Boivert to see if he can provide a new 
version for this jar (we are currently using 1.0.0) and he replied that 
he will try to find some time in a very busy schedule to deliver a 
1.0.1, which will have to be deployed to a maven repo. Another coiple of 
days...

Here is what I suggest at this point : I have created a sub-project 
(apacheds-jdbm) containing the JDBM code with almost all of the tests 
(except the performance tests), and it works pretty well. I will commit 
this code and my modification in the bigbang branch, as I have to move 
on to the next step : removing the JNDI layer.

The reason why I'm impatient to do that is that I have a 13 000 lines 
long diff pending, and I'm afraid to do some modifications on the server 
with around 100 impacted files, as it may break the currently working 
server...

FYI, I have run some perf test on my laptop, and here are the compared 
results for 1.5.2 and bigband (as always, local test, not meaningfull, 
blah blah, for your eyes only :)

1.5.2       : 3506 random search req/s
bigbang : 4296 random search req/s

I also profiled the server and found interesting numbers :

LdapDN handling : 37.6% of the CPU (out of which around 18% can be 
squeeze when the JNDI layer will have been removed)

ServerEntry handling : 19.60% of the CPU

ASN.1 codec : 10,15%

This is 67,4% of all the consumed CPU in tasks which are difficult to 
improve (if we except the 18% we can squeeze from some useless double 
LdapDN parsing), the rest is spreaded in small methods.

A blind guess is that it will be quite challenging to gain a lot now. My 
own expectation is that we should be able to go up to 6000 searches 
req/s on my laptop, but then, we will have to check the LdapDN parsing 
again.

It would be very cool if someone can double check the ServerEntry clone 
method to see if it's optimal, and if we can improve its code (or even 
find some bug or bad logic in it).

I was also thinking that some lazzy cloning might be enough (ie, cloning 
only the attribute container, not the attribute values themselves, as in 
many case, we are just removing attributes from the entry before 
returning it, keeping the attributes unmodified. That would save a few %)

That's it, I'm now waiting for Eris to be up and running...

thanks for your attention !

-- 
--
cordialement, regards,
Emmanuel L�charny
www.iktek.com
directory.apache.org