Mailing-List: contact dev-help@directory.apache.org; run by ezmlm
Precedence: bulk
Reply-To: "Apache Directory Developers List" <dev@directory.apache.org>
Received-SPF: pass (asf.osuosl.org: domain of aok123@bellsouth.net designates
 205.152.59.66 as permitted sender)
Message-ID: <44CA1E6E.3090305@bellsouth.net>
Date: Fri, 28 Jul 2006 10:25:50 -0400
From: Alex Karasulu <aok123@bellsouth.net>
User-Agent: Thunderbird 1.5.0.4 (X11/20060615)
MIME-Version: 1.0
To: Apache Directory Developers List <dev@directory.apache.org>
Subject: Re: LDAP codec streaming
References: <f275eef10607271847u464898ccgafab0a0bbd6efedb@mail.gmail.com>
In-Reply-To: <f275eef10607271847u464898ccgafab0a0bbd6efedb@mail.gmail.com>
Content-Type: text/plain; charset=ISO-8859-1; format=flowed
Content-Transfer-Encoding: 7bit

Norbet Reilly wrote:
> Hi everyone,
> 
> Thanks for my inclusion to the project as a "mentored commiter" (I
> didn't mind "proby" myself ;-), it's an honour to be involved with
> such an exciting opensource project.

Well it's just good enough to call you a committer along with the 
others.  We just want you to be careful in the beginning.

> I was just wanting to ask one of the ADS gurus what happens exactly at
> the LDAP protocol level when an extremely large result set is returned
> by ADS for an LDAP search(). Is a client able to start retreiving
> results via their NamingEnumeration before the server has completed
> its search? I'm hoping there is some sort of "chunking" at the
> protocol level so that batches of results are streamed to the client
> as they become available, rather then the waiting for the entire
> result set to be available first. Is this the case?

Yes and no :).  There is no chunking of an individual PDU sent back. 
Meaning each SearchEntryResponse prepared then sent out the door.  If an 
entry you're returning has a 1 mb jpeg in it then this will be used to 
prepare a response and return it rather than streaming out the jpeg. 
Old incarnations of MINA used to allow us to do this but it was complicated.

For the search operation, we do not collect all entries before sending a 
response.  That would kill the server.  ApacheDS will pull out and send 
a result entry to you one at a time.  MINA may lag a little bit and some 
results may actually sit a while before going out the door.  But for all 
practical purposes we pull from the db and pump out results one by one.

The way it works is a bit complicated and will take some time to 
discuss.  I can go into it if enough people are interested and willing 
to document it :) after I describe how she does this.

Alex