Mailing-List: contact dev-help@directory.apache.org; run by ezmlm
Precedence: bulk
Reply-To: "Apache Directory Developers List" <dev@directory.apache.org>
Received-SPF: pass (athena.apache.org: domain of elecharny@gmail.com
 designates 209.85.212.173 as permitted sender)
Message-ID: <52C26EC6.7020701@gmail.com>
Date: Tue, 31 Dec 2013 08:14:14 +0100
From: =?UTF-8?B?RW1tYW51ZWwgTMOpY2hhcm55?= <elecharny@gmail.com>
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.7;
 rv:24.0) Gecko/20100101 Thunderbird/24.2.0
MIME-Version: 1.0
To: Apache Directory Developers List <dev@directory.apache.org>
Subject: Re: Scalability / performance testing?
References: 
 <CAJEX5ou5+3j3C2RM0jG9aw=B+9oM4tnMhgVpo55F1s1U6-44eQ@mail.gmail.com>
	<52C1A960.7000200@gmail.com>
 <CAJEX5os49eDefZ=zv9uO-oG9bHi=hFk+8Yx6DE=JpVu5V5nbew@mail.gmail.com>
In-Reply-To: 
 <CAJEX5os49eDefZ=zv9uO-oG9bHi=hFk+8Yx6DE=JpVu5V5nbew@mail.gmail.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

Le 12/30/13 7:57 PM, Marc Boorshtein a écrit :
>>
>>
>> I have done some benchmarks 6 months ago with 5 injectors sending as
>> manu search request as possible to the server. I won't say it's enough
>> to guarantee that the server is stable enough...
>>
> Performance testing is an art, not a science :-)

Or maybe science-fiction...
>
>
>> This is typical of a client hving closed the connection just after
>> having sent a search request. It sounds weird, but it's a possibility :
>>
>>     public final void handleMessage( IoSession session, T message )
>> throws Exception
>>     {
>>         LdapSession ldapSession =
>> ldapServer.getLdapSessionManager().getLdapSession( session );
>>
>>         if ( ldapSession == null )
>>         {
>>             // in some cases the session is becoming null though the
>> client is sending the UnbindRequest
>>             // before closing
>>             LOG.info( "ignoring the message {} received from null
>> session", message );
>>             return;
>>         }
>>
>>
> OK, so like if there was a timeout on the client end?  I send the request,
> but it takes too long so it closes the connection before the search is done
> being processed?

That's an option. We added this check to avoid such situation :
https://issues.apache.org/jira/browse/DIRSERVER-1404

>
>
>
>>
>> The BIND operation is different from a session. A LdapSession is
>> associated with a physical connection, while the Bind operation is just
>> a message sent by the user. For some reason, (the protocol mandate it),
>> you have to accept concurrent operations for a single session, like for
>> instance an Abandon request should be processed even if a search request
>> is being processed.
>>
>> Thus we have more than one thread processing the incoming requests, and
>> in some cases, processing a Unbind request might be unliky and being
>> done on a already closed connection.
>>
>
> Is there a way to tell if requests are coming in on one connection or over
> multiple connections?

A request always come from one single conection. It's not possible that
2 requests for the same session are arriving from 2 different
connections. What happens internally is that requests are processed by a
pool of thread, which makes it possibe for 2 requests to be processed in
parallel.


>
>>> Are there any guide lines for how to configure apacheds?  ie the number
>> of
>>> threads?
>> You can increase the number of processing thread in the transport layer
>> :
>>
>> http://directory.apache.org/apacheds/advanced-ug/2-server-config.html#transports
>> (check the ads-transportNbThreads paramenter).
>>
>>
> Great, do you have any guidance on how to set these options?  ie is it
> better to have n threads where n is the number of cores/processors?  Whats
> the relationship between ads-transportBackLog and memory/processing power?

As a rule of thumb, nb of threads = nb of processors/core +1. That
should be enough.
>
>
>
>> Now, if you can insolate some problem, we would be happy to check what's
>> going on !
>>
>>
> Sure.  Right now I'm just trying to understand what the logs are telling
> me.  MyVD's performance use case is probably very different from "straight"
> apacheds.  These tests seem to take between 300ms and 1.5s for
> authentication (this isn't 1-1 with a single LDAP operation.  Its actually
> several.) where as ApacheDS the data is all local so the timings are
> different.
>
>
>
>> One last thing : we will conduct some load tests very soon.
>>
>>
> We'd be happy to donate compute cycles on ec2 and assist in the testing
> process however we can!


What I would be interested in is to run the exact tests you are running.
But that will be no sooner than next week, as I'm on a little trip for
the next 2 days (and my brain might be a bit too fried in the 2 or 3
days after those two days to be useful at all...)

Have a happy new year in the mean time !


-- 
Regards,
Cordialement,
Emmanuel Lécharny
www.iktek.com