lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Karsten Konrad" <Karsten.Kon...@xtramind.com>
Subject AW: Problem with long run IndexSearcher
Date Mon, 19 May 2003 16:16:58 GMT

Hi,

>>
I'm not sure what impact my strategy will have if your application pages through hit 
results over a period of time...might not work at all (or perhaps inconsistently).
>>

my solution is as follows: I use a class that encapsulates the search result and
that has a reference to the searcher object it relies on. Another class holds
the IndexSearcher, say SearcherContainer. If I want to replace the old IndexSearcher 
by new ones, I replace this SearcherContainer instance. The SearcherContainer tries to 
close its IndexSearcher in its finalized() method. By this, the closing of all 
outdated searchers is done by the garbage collector if appliable. 

As long as any user has search results to display in his session object (i.e., is browsing
through some search results), the IndexSearcher will not be closed and the
search results stay valid. But if all users use new SearcherContainers, the old ones can
be destroyed by garbage collection and hence all IndexSearchers will closed in the
end. There seems to be no problem to use both outdated and current IndexSearcher objects 
at the same time, the behavior looks totally consistent to the user.

I think the whole appproach could be much simpler if every reader would simply
close itself within its finalized method, but nevertheless it works this way quite
well. In contrast to this, generating the presentation for the hit results in
advance or redoing searches is more expensive. 

Karsten

-----Urspr√ľngliche Nachricht-----
Von: Eric Isakson [mailto:Eric.Isakson@sas.com]
Gesendet: Montag, 19. Mai 2003 16:53
An: Lucene Developers List
Betreff: RE: Problem with long run IndexSearcher


I was just reading Karsten's note, I'm not sure what impact my strategy will have if your
application pages through hit results over a period of time...might not work at all (or perhaps
inconsistently). The application I use recalculates search  results for each page as I page
through hits rather than trying to hold onto the results from page to page. If the reader
changed to an index that resulted in different hits it would seem inconsistent. You would
only notice the inconsistency if you happened to be paging around while my reader was being
refreshed :(

Hmph, guess I didn't really solve the problem with this new class, but it is a start on a
solution and I'd be happy to keep working.

Eric 

-----Original Message-----
From: Giulio Cesare Solaroli [mailto:slrgcsa@ibn-italy.com] 
Sent: Monday, May 19, 2003 10:44 AM
To: Lucene Developers List
Subject: Re: Problem with long run IndexSearcher


Hi Eric,

On Monday, May 19, 2003, at 16:18 Europe/Rome, Eric Isakson wrote:

> bah, accidently sent that before I was finished typing...
>
> see:
> http://nagoya.apache.org/eyebrowse/ReadMsg?listName=lucene-
> user@jakarta.apache.org&msgNo=1859
>
> which really should be added to the FAQ.
>
> It just so happens that I spent a good deal of my weekend working on a
> class to solve this problem. It hasn't been tested much and I'm new to  
> writing apps that deal with threading issues (so I don't know if I  
> blundered anywhere). I'd love to see if it works and get feedback from  
> anyone that uses it.

Great!!!

> See my message at:
>
> http://nagoya.apache.org/eyebrowse/ReadMsg?listName=lucene-
> dev@jakarta.apache.org&msgNo=3393

I did read it, but not carefully enught to realize it was what I was  
looking for. I will give it a try!!

> If you have any feedback, pleass attach comments to the bug.

Ok. I will let you know.

Thanks again for your support.

Giulio Cesare



> Eric
>
> --
> Eric D. Isakson        SAS Institute Inc.
> Application Developer  SAS Campus Drive
> XML Technologies       Cary, NC 27513
> (919) 531-3639         http://www.sas.com
>
>
>
>
> -----Original Message-----
> From: Eric Isakson [mailto:Eric.Isakson@sas.com]
> Sent: Monday, May 19, 2003 10:05 AM
> To: Lucene Developers List
> Subject: RE: Problem with long run IndexSearcher
>
>
> Giulio,
>
> This problem stems from the fact that the IndexReader won't see your
> changed index.
>
> See the faq entries:
>
>
> -----Original Message-----
> From: Giulio Cesare Solaroli [mailto:slrgcsa@ibn-italy.com]
> Sent: Monday, May 19, 2003 9:38 AM
> To: lucene-dev@jakarta.apache.org
> Subject: Problem with long run IndexSearcher
>
>
> Hi all,
>
> first let me express my compliments for Lucene.
> I have been up for a full week-end to double check the results I was 
> having because I couldn't belive what I saw; with a stupid application 
> I could index DB data at a sustained rate of 50 documents per second.
>
> Now we have more that 2 millions documents indexed and the performance 
> are still excellent; our main bottle neck is still the DB.
>
> Our situation:
> - we are indexing new documents at a sustained rate (an average of 
> 40.000 new documents a day);
> - we have written a small xmlRpc server in Java to search the index 
> from other applications.
>
> The xmlRpc server creates a single instance of IndexSearcher a reuse 
> it for each query issued. For each request, a new Query object is 
> created and the documents found are returned to the client.
>
> The problem we are seeing is that the documents indexed after the 
> xmlRpc server is started will not be found until the server is 
> restarted.
>
> Is this our foult, or the way IndexSearcher should work?
>
> What is the best way to keep the IndexSearcher up to date with the 
> updated index?
>
> Thanks for your attention,
>
> Giulio Cesare Solaroli
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: lucene-dev-unsubscribe@jakarta.apache.org
> For additional commands, e-mail: lucene-dev-help@jakarta.apache.org
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: lucene-dev-unsubscribe@jakarta.apache.org
> For additional commands, e-mail: lucene-dev-help@jakarta.apache.org
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: lucene-dev-unsubscribe@jakarta.apache.org
> For additional commands, e-mail: lucene-dev-help@jakarta.apache.org
>


---------------------------------------------------------------------
To unsubscribe, e-mail: lucene-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: lucene-dev-help@jakarta.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: lucene-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: lucene-dev-help@jakarta.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: lucene-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: lucene-dev-help@jakarta.apache.org


Mime
View raw message