directory-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Alex Karasulu <akaras...@gmail.com>
Subject Re: ldap server buffers search results?
Date Sun, 05 Apr 2009 19:44:24 GMT
On Sun, Apr 5, 2009 at 11:57 AM, Emmanuel Lecharny <elecharny@apache.org>wrote:

> Alex Karasulu wrote:
>
>> On Sat, Apr 4, 2009 at 6:41 PM, Emmanuel Lecharny <elecharny@apache.org
>> >wrote:
>>
>>
>>
>>> Some follow-up :
>>>
>>> I have done many tests, and I successfully got a working server without
>>> having all the messages stored in memory, simply by changing the way the
>>> ExecutorFilter is instanciated. In LdapService, line 361, instead of :
>>>
>>>      // Now inject an ExecutorFilter for the write operations
>>>      // We use the same number of thread than the number of IoProcessor
>>>      // (NOTE : this has to be double checked)
>>>      ((DefaultIoFilterChainBuilder)chain).addLast( "executor",
>>>              new ExecutorFilter( new OrderedThreadPoolExecutor(
>>> getTcpTransport().getNbThreads() ),
>>>                  IoEventType.WRITE ) );
>>>
>>> use :
>>>
>>>      // Now inject an ExecutorFilter for the write operations
>>>      // We use the same number of thread than the number of IoProcessor
>>>      // (NOTE : this has to be double checked)
>>>      ((DefaultIoFilterChainBuilder)chain).addLast( "executor",
>>>              new ExecutorFilter( new OrderedThreadPoolExecutor(
>>> getTcpTransport().getNbThreads() ) ) );
>>>
>>> Now, the executor is used in both ways, and it seems to work much better.
>>>
>>>
>>>
>>
>> So the Executor is not being used for all event types: reads, and writes.
>> How is this effecting the dynamic.  I think it's very important for us to
>> have a good written characterization of the problem before hand.  Then we
>> can follow up with a summary of what each approach does.
>>
>>
> There is a bit of an issue : you can just have one single executor in the
> chain. Right now, I'm analysing the MINA internals to see what are the
> reason why limitating the IoEvent to WRITE only leads to OOM, or at least,
> late sending of all the messages, when accepting all the events is just
> ok...
>
>> The reason why I'm being a bit anal about this is because I know the
>> dynamics of MINA are going to be dramatically impacted when ApacheDS is
>> put
>> under heavy load.
>>
>>
> I'm also really scared about what could happen in a real environment (ie
> thousands of clients, with some of them sending more than one request at the
> same time).
>
> This is painful ...
>
>
Oh yes I know it must be very painful but it is perhaps the most important
work anyone can be doing for ApacheDS.

Alex

Mime
View raw message