directory-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Alex Karasulu" <akaras...@apache.org>
Subject Re: [ApacheDS] Recommendations for implementing search time limits
Date Wed, 20 Aug 2008 00:14:52 GMT
I just made some changes to Cursors which should be made regardless of the
route we decide to take here.  I went back and made sure Cursors break out
of loops in advance methods as well as throw exceptions when get() is called
when the Cursor has been closed.  This is just a sound thing to do.

Secondly I added a new overload to the close() method on Cursor.  This
method takes an Exception argument called the reason.  If this overload is
used and a non-null Exception argument is provided then that exception is
thrown instead of the standard CursorClosedException.  This is a good idea
in general because it allows Cursor closing threads to communicate with
Cursor using threads.

The Cursor might be closed due to it being finished with, an LDAP operation
being abandoned, search limits being reached or any other reason and you
want the using thread to know why sometimes.  The exception used can help
communicate that.

Alex

On Tue, Aug 19, 2008 at 6:29 PM, Alex Karasulu <akarasulu@apache.org> wrote:

> Hi all,
>
> I have an interesting situation to deal with while trying to find a way to
> implement search time limits.  I'm a bit perplexed on how best to implement
> this and so any suggestions are great.  Here's a quick breakdown of the
> problem.
>
> Search requests must adhere to search time limits.  The request is used to
> build a system of nested Cursors and evaluators similar in structure to the
> search filter.  The final Cursor produced will return candidates accepted by
> the filter expression.  Cursors can be advanced using first(), last(),
> next(), and previous() methods.  While advancing a loop performs evaluations
> to find the first, last, next or previous candidate matching the filter.
> Those entries not matching the filter a not returned.  It's a bit more
> complex than this but this trivial view is sufficient to discuss the
> problem.
>
> So we have to stop searching after some time limit.  Previously before the
> bigbang we used to just check the time limits high up in the code (in the
> search handler) after getting each candidate.  This did not always work so
> well because sometimes with very large directories and poor search filters
> it would take a while to find the next candidate to return so we would
> overshoot the time limit.   The problem is primarily due to these evaluation
> loops in the Cursor while trying to advance to a candidate.  While in this
> loop, going over entries that do not match, there is no way for the Cursor
> to detect that time is up.
>
> There are a few ways I thought we can solve this problem.  First these
> loops will have to check if the time is up between iterations either
> directly or indirectly through some parameter.  It would be nice if the
> Cursor can detect this on it's own but tunnelling the size limits deep down
> through this system of nested Cursors during construction is a real PITA.  I
> was thinking of using the closed/open state of the Cursor to naturally halt
> these loops.  This should happen anyway.  The only problem is another thread
> is needed in the SearchHandler to close the Cursor when time limits are
> exceeded.
>
> I talked to Emmanuel about this on IM and he preferred the approach where a
> Cursor is passed the time limits and handles timing out on it's own.  I'm
> not too keen on this idea because a Cursor by it's nature is not and may not
> be time limited.  However it does need to respond to being closed
> appropriately even while one thread is advancing it.  I think Emmanuel
> rightfully so dislikes the idea of having a timer thread (really a
> schedulable executor of tasks for closing search Cursors) in the search
> handler.  Don't know if others see a better option here or if one way is
> more preferrable than the other.
>
> Thoughts?
>
> Alex
>
> --
> Microsoft gives you Windows, Linux gives you the whole house ...
>



-- 
Microsoft gives you Windows, Linux gives you the whole house ...

Mime
View raw message