directory-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Emmanuel Lecharny (JIRA)" <>
Subject [jira] Commented: (DIRSERVER-1161) search results are not streamed to the client until final done response is queued
Date Fri, 22 Aug 2008 07:32:44 GMT


Emmanuel Lecharny commented on DIRSERVER-1161:

We have discussed the issue further with Alex this week, and we think that the problem should
be fixed on ADS, not in MINA.

However, the fix won't be a quick and easy one. Here is what we discussed about :
- considering that if the client is slow, we may create as many Message stored in memory as
we have entries to send back
- and that we don't want to block a thread waiting for a blocking queue, as we have a limited
number of thread to process LDAP requests
- we have to find a way to be informed when the socket is ready to accept a new write, and
then pick a thread in an executor to get the next entry to write (maybe the next N entries,
to avoid some expensive calls to the executor)
- that also mean we must be able to push the cursor we have just created into a session container,
and free the thread used to create this cursor

We have to think more about the details, now.

> search results are not streamed to the client until final done response is queued
> ---------------------------------------------------------------------------------
>                 Key: DIRSERVER-1161
>                 URL:
>             Project: Directory ApacheDS
>          Issue Type: Bug
>          Components: core
>    Affects Versions: 1.0
>         Environment: JDK 1.5.0_11 
>            Reporter: Norval Hope
>            Assignee: Emmanuel Lecharny
>            Priority: Blocker
>             Fix For: 1.5.4
>         Attachments: apacheds_1.5.1_streaming.patch, apacheds_1.5.1_streaming.patch,
apacheds_1.5.1_streaming_log_output.txt, installers_1.5.1_streaming.patch, installers_1.5.1_streaming.patch,
mina_1.1.2_streaming.patch, mina_1.1_trunk_streaming.patch, pom.xml, streaming_log_output.txt,
>   Original Estimate: 120h
>  Remaining Estimate: 120h
> Search results accumulate in Events on the SessionBuffer.eventQueue within ExecutorFilter.fireEvent()
until final done response for the search is written to the session and then all results for
the search (possibly millions depending on the search and state of the directory) are written
out at once. This is a big problem for scalability and I gather from previous correspondence
with Alex that this behaviour is unexpected.

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message