Mailing-List: contact dev-help@directory.apache.org; run by ezmlm
Precedence: bulk
Reply-To: "Apache Directory Developers List" <dev@directory.apache.org>
Received-SPF: pass (nike.apache.org: domain of elecharny@gmail.com designates
 209.85.219.180 as permitted sender)
DomainKey-Signature: a=rsa-sha1; c=nofws;
        d=gmail.com; s=gamma;
        h=sender:message-id:date:from:user-agent:mime-version:to:subject
         :references:in-reply-to:content-type:content-transfer-encoding;
        b=PWTK3jUGNqVvBxDaYOGJUcppnd8S7M1ap65dmDqODAzsJgNdna9UYgSYLhmpw2RoCO
         0n8zWhMZKFA0xgoAuRwIXejew+UpHG57SriA1cCAVfDZraIAJhc+S0Xn4OKK0BSfdsrs
         TwyXHoA9jBdR1nDIwWZayMemIH4cl4FSnAjfg=
Sender: Emmanuel Lecharny <elecharny@gmail.com>
Message-ID: <49D8D4E5.7050606@nextury.com>
Date: Sun, 05 Apr 2009 17:57:25 +0200
From: Emmanuel Lecharny <elecharny@apache.org>
User-Agent: Thunderbird 2.0.0.21 (X11/20090318)
MIME-Version: 1.0
To: Apache Directory Developers List <dev@directory.apache.org>
Subject: Re: ldap server buffers search results?
References: <debd1dad0904021410h74f225d3kece23a916204de44@mail.gmail.com>
	 <49D5EEFC.7030703@planetquake.com>
	 <debd1dad0904040928t32cdc64dyca6af8d956668cca@mail.gmail.com>
	 <49D7E228.7070508@nextury.com>
 <a32f6b020904050658p24573c30u3115d00c60782314@mail.gmail.com>
In-Reply-To: <a32f6b020904050658p24573c30u3115d00c60782314@mail.gmail.com>
Content-Type: text/plain; charset=ISO-8859-1; format=flowed
Content-Transfer-Encoding: 8bit

Alex Karasulu wrote:
> On Sat, Apr 4, 2009 at 6:41 PM, Emmanuel Lecharny <elecharny@apache.org>wrote:
>
>   
>> Some follow-up :
>>
>> I have done many tests, and I successfully got a working server without
>> having all the messages stored in memory, simply by changing the way the
>> ExecutorFilter is instanciated. In LdapService, line 361, instead of :
>>
>>       // Now inject an ExecutorFilter for the write operations
>>       // We use the same number of thread than the number of IoProcessor
>>       // (NOTE : this has to be double checked)
>>       ((DefaultIoFilterChainBuilder)chain).addLast( "executor",
>>               new ExecutorFilter( new OrderedThreadPoolExecutor(
>> getTcpTransport().getNbThreads() ),
>>                   IoEventType.WRITE ) );
>>
>> use :
>>
>>       // Now inject an ExecutorFilter for the write operations
>>       // We use the same number of thread than the number of IoProcessor
>>       // (NOTE : this has to be double checked)
>>       ((DefaultIoFilterChainBuilder)chain).addLast( "executor",
>>               new ExecutorFilter( new OrderedThreadPoolExecutor(
>> getTcpTransport().getNbThreads() ) ) );
>>
>> Now, the executor is used in both ways, and it seems to work much better.
>>
>>     
>
> So the Executor is not being used for all event types: reads, and writes.
> How is this effecting the dynamic.  I think it's very important for us to
> have a good written characterization of the problem before hand.  Then we
> can follow up with a summary of what each approach does.
>   
There is a bit of an issue : you can just have one single executor in 
the chain. Right now, I'm analysing the MINA internals to see what are 
the reason why limitating the IoEvent to WRITE only leads to OOM, or at 
least, late sending of all the messages, when accepting all the events 
is just ok...
> The reason why I'm being a bit anal about this is because I know the
> dynamics of MINA are going to be dramatically impacted when ApacheDS is put
> under heavy load.
>   
I'm also really scared about what could happen in a real environment (ie 
thousands of clients, with some of them sending more than one request at 
the same time).

This is painful ...

-- 
--
cordialement, regards,
Emmanuel L�charny
www.iktek.com
directory.apache.org