directory-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Stefan Seelmann <seelm...@apache.org>
Subject Re: [PagedSearch] Cleanup strategy
Date Wed, 17 Dec 2008 13:19:18 GMT

Quoting Emmanuel Lecharny <elecharny@gmail.com>:

> Hi,
>
> this is the last step to handle the paged search request : cleaning the
> remaining paged search when they are not used.
>
> The problem is that a client can launch a paged search, get some few
> pages, and then stop requesting the remaining pages, without closing
> the request. Obviously, this is bad, bad, bad, but we won't be able to
> spank all the clients ;)
>
> The created paged search context will then remain until the session is
> closed, keeping all the associated data (cursors, and such) in memory.
> Plus the client may launch more than one paged search request, leading
> to a quick OOM...
>
> There are a few strategy we can implement to avoid such a problem :
> 1) first, we can limit the number of paged search request per session.
> I guess that a session with more than  5 or 10 opened paged search is
> very unlikely to occur. I suggest that we limit the number of opened
> paged searches to 5 atm (of course, this should be configurable), and
> unlimited for the administartor. When we reach this number, then the
> oldest paged search will be removed from the session.

Maybe a per-user limit would also make sense. A user could open  
multiple sessions in order to start more paged searches. Also a global  
limit could make sense. But we should keep it simple ;-)

Another question is how to define the "oldest" paged search. Do we use
- either the timestamp when the initial search were started and the  
first page was requested
- or the timestamp when the last page was requested?

> 2) we can also manage a timeout for the paged search. That mean a
> thread will be in charge of removing all the time-outed paged search
> across the sessions.

Do we already have another thread for clean-up stuff? Maybe we could  
plug into such an existing thread.

If the clean-up isn't too time consuming maybe we could do such checks  
in other requests. e.g. each time when a paged search request comes in  
we could check that. This would avoid an extra thread.

> 3) We can also wait until the session dies, and cleanup the associated
> paged search.
>
> In any case, we must do something, as the cursors _have_ to be closed.
>
> Does those 3 proposals make sense to you? Did I forgot something? Do
> you see a better strategy ?

The server also implements persistent search. Are there similar  
problems with open resources? Maybe we can look how it is solved there?

Regards,
Stefan



Mime
View raw message