Mailing-List: contact dev-help@directory.apache.org; run by ezmlm
Precedence: bulk
Reply-To: "Apache Directory Developers List" <dev@directory.apache.org>
Received-SPF: pass (athena.apache.org: domain of elecharny@gmail.com
 designates 209.85.134.185 as permitted sender)
DomainKey-Signature: a=rsa-sha1; c=nofws;
        d=gmail.com; s=gamma;
        h=message-id:date:user-agent:mime-version:to:subject:references
         :in-reply-to:content-type:content-transfer-encoding:from;
        b=tIYdNxOmpXIVF4fk3JkCLLy+8VWUyb5sXNfO+y/89yyaiA0ehEkHFMaXDIcLthU2vx
         2NlDeQyhnIuIqXUmm8t3Cl9bukQ3d4z1hd+HdYq6iQOfHcFJprQLLFYL2W3pnXV6ydxD
         DMYB/WvGh1fCBBGvmV2iJmkgdxdqNZ+f3OlBk=
Message-ID: <49441E36.3090100@nextury.com>
Date: Sat, 13 Dec 2008 21:42:30 +0100
User-Agent: Thunderbird 2.0.0.18 (X11/20081125)
MIME-Version: 1.0
To: Apache Directory Developers List <dev@directory.apache.org>
Subject: Re: Paged Search Control questions
References: <4943B7B3.6040604@nextury.com> <49441BA3.6070703@apache.org>
In-Reply-To: <49441BA3.6070703@apache.org>
Content-Type: text/plain; charset=ISO-8859-1; format=flowed
Content-Transfer-Encoding: 8bit
From: Emmanuel Lecharny <elecharny@gmail.com>

Stefan Seelmann wrote:
> Emmanuel Lecharny schrieb:
>   
>> Hi guys,
>>
>> as I'm busy implenting this control, 
>>     
>
> Cool
>
>   
>> 1) considering that we have a server sizeLimit, a request sizeLimit and
>> a page size limit, I'm wondering if we can simply ignore the request
>> size limit. The page size limit can change, even if the paged result is
>> being processed, but the RFC says "If the page size is greater than or
>> equal to the sizeLimit value, the server should ignore the control as
>> the request can be satisfied in a single page". Should I consider that
>> the 'sizeLimit' is the request sizeLimit ? My personnal bet is : yes.
>>     
>
> Yes, I would say the 'sizeLimit' is the request sizeLimit.
>
> I think we should not ignore the requst sizeLimit, I would consider it
> as client-side limit for the complete search over all pages.
>
> Say the request sizeLimit is 10 and the page size is 8 for both requests
> then the first result contains 8 entries and the second contains 2
> entries plus a LDAP error #4.
>   
Makes perfect sense. So we will have to remember the number of already 
returned entries, and compare it
to the sizeLimit. Fine.
>> 3) regarding the search request immutability : it's pretty hard to check
>> that the filter hasn't changed, as it may be a complex one, with a
>> different structure and a a different order. I think that this
>> constraint is fully absurd, as the client will obviously create one
>> request, and send a null cookie every time it will send a new paged
>> search, so I don't see the validity of such a check. Nevertheless,
>> should we try to implement such a check ? My personal guess, again, is
>> that it's useless.
>>     
>
> To check if the filter changed logically is really complex. I think you
> could just check if the string representation or the bytes the are
> received were changed.
>   
This is what I currently do. There is nothing more I can do...
>   
>> I'm also interested to have some feedback about how this control is
>> handled by the other ldap servers, considering the many factors
>> influencing this control :
>> - internal server size limit
>>     
>
> Here different implementations are different.
> - OpenLDAP just stops if the server side limit is exceeded.
>   
I thought you can define soft and hard limit... I have to check in my 
favorite OpenLDAP book 'Mastering OpenLDAP'.
> - For MS AD has a default server side limit of 1000, but when this
> control is used you could get more
>   
Which violates the spirit of this limit... Thanks, AD !
>   
>> - how many of such paged search can be handled for a single client
>>     
> Infinite?
>   
Costly ... Currently, we don't have any limit, but at some point, we 
will need to discard some old searches, as each of them eat a big chunk 
of memory. I would say that a circular list of 10 paged search request 
should be a valid default. It may be configurable, too.

The main issue I have atm is that I have to implement a mechanism to 
discard timeout-ed requests.
>   
>> - what happens when we send a bad cookie to the server
>>     
> Either start a new search or return an error
>   
I think that sending an error is better.
>   
>> - what happen when we play with the sizeLimit parameter
>>     
> You mean the request sizeLimit? If it is changed it should be considered
> as a new search.
>   
You're right !
>   
>> Last, not least, as we are using cursors to get the entries from the
>> backend, we are able to move forward or backward. It would be
>> interesting to extend this control to allow a backward pagedSearch (for
>> instance, providing a negative paged size). Would it be interesting ?
>>     
>
> There is another control for this: VLV, see [1]. But is also need
> server-side sorting.
>   
True ! I forgot about this guy...

Thanks Stefan, very interesting responses ! I was running in circles, I 
start to see the light now :)

-- 
--
cordialement, regards,
Emmanuel L�charny
www.iktek.com
directory.apache.org