directory-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Norval Hope (JIRA)" <j...@apache.org>
Subject [jira] Commented: (DIRSERVER-1161) search results are not streamed to the client until final done response is queued
Date Thu, 10 Apr 2008 02:18:05 GMT

    [ https://issues.apache.org/jira/browse/DIRSERVER-1161?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12587463#action_12587463
] 

Norval Hope commented on DIRSERVER-1161:
----------------------------------------

On futher investigation my comments 1 and 3 above are spurious. We're doing work on 2. to
see how much peak memory use that saves us, as a work-around.

My gut feeling is that the lack of streaming is due to two threads interacting with the same
underlying session, where we have the pool-3-thread-4 which got the messageReceived() and
is therefore invoking SearchHandler and a SocketAcceptorIoProcessor-0.0 thread which is writing
data that never seems to be sent to the client despite being written and flushed. It's like
somehow the "result response" message causes the underlying session to change for a reading
state to a state where the buffered output data can finally be sent off to the client, although
through all my stepping through the MINA filter chains etc in the debugger I can't see the
magic moment where this state transition occurs. It doesn't seem to relate to the state of
the thread invoking the SearchHandler  as best I can see (i.e. like a lock being held onto),
but rather to something that happens due to it writing the final response done message to
the session which triggers some change in the  SocketAcceptorIoProcessor-0.0 thread.

> search results are not streamed to the client until final done response is queued
> ---------------------------------------------------------------------------------
>
>                 Key: DIRSERVER-1161
>                 URL: https://issues.apache.org/jira/browse/DIRSERVER-1161
>             Project: Directory ApacheDS
>          Issue Type: Bug
>          Components: core
>    Affects Versions: 1.0
>         Environment: JDK 1.5.0_11 
>            Reporter: Norval Hope
>         Attachments: streaming_log_output.txt, streaming_logging.patch
>
>   Original Estimate: 120h
>  Remaining Estimate: 120h
>
> Search results accumulate in Events on the SessionBuffer.eventQueue within ExecutorFilter.fireEvent()
until final done response for the search is written to the session and then all results for
the search (possibly millions depending on the search and state of the directory) are written
out at once. This is a big problem for scalability and I gather from previous correspondence
with Alex that this behaviour is unexpected.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message