directory-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Alex Karasulu <>
Subject Re: LDAP codec streaming
Date Fri, 28 Jul 2006 14:25:50 GMT
Norbet Reilly wrote:
> Hi everyone,
> Thanks for my inclusion to the project as a "mentored commiter" (I
> didn't mind "proby" myself ;-), it's an honour to be involved with
> such an exciting opensource project.

Well it's just good enough to call you a committer along with the 
others.  We just want you to be careful in the beginning.

> I was just wanting to ask one of the ADS gurus what happens exactly at
> the LDAP protocol level when an extremely large result set is returned
> by ADS for an LDAP search(). Is a client able to start retreiving
> results via their NamingEnumeration before the server has completed
> its search? I'm hoping there is some sort of "chunking" at the
> protocol level so that batches of results are streamed to the client
> as they become available, rather then the waiting for the entire
> result set to be available first. Is this the case?

Yes and no :).  There is no chunking of an individual PDU sent back. 
Meaning each SearchEntryResponse prepared then sent out the door.  If an 
entry you're returning has a 1 mb jpeg in it then this will be used to 
prepare a response and return it rather than streaming out the jpeg. 
Old incarnations of MINA used to allow us to do this but it was complicated.

For the search operation, we do not collect all entries before sending a 
response.  That would kill the server.  ApacheDS will pull out and send 
a result entry to you one at a time.  MINA may lag a little bit and some 
results may actually sit a while before going out the door.  But for all 
practical purposes we pull from the db and pump out results one by one.

The way it works is a bit complicated and will take some time to 
discuss.  I can go into it if enough people are interested and willing 
to document it :) after I describe how she does this.


View raw message