lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Chris Lamprecht <clampre...@gmail.com>
Subject Re: Search Performance
Date Fri, 18 Feb 2005 21:03:29 GMT
I recently dealt with the issue of re-using a Searcher with an index
that changes often.  I wrote a class that allows my searching classes
to "check out" a lucene Searcher, perform a search, and then return
the Searcher.  It's similar to a database connection pool, except that
all clients can share the same Searcher (I don't think there is any
benefit to keeping a true "pool" and giving a different Searcher to
each client -- someone let me know if this is incorrect).

So I just keep a reference count to my Searcher, which gets
incremented at checkout and decremented at checkin.  So the logic is
approximately:

initialize lastVersion to -1

checkout:
if (lucene index version != lastVersion) {
   create a new IndexSearcher and update lastVersion
} 
refcount++;
return the searcher

And on checkin:
refcount--;
if (refcount ==0 and there is a newer lucene index version) {
   close the searcher being checked in
}


Of course there are some more details to keep info on the open
searchers, make it thread-safe, etc.  I also plan to only check for a
new index if some minimum time threshold has passed (5 minutes or so).
 I'd be interested in hearing others' solutions/patterns for this.

-Chris

On Fri, 18 Feb 2005 11:57:32 -0500, Michael Celona
<mcelona@criticalmention.com> wrote:
> My index is changing in real time constantly... in this case I guess this
> will not work for me.... any suggestions...
> 
> Michael
> 
> -----Original Message-----
> From: David Townsend [mailto:david.townsend@magus.co.uk]
> Sent: Friday, February 18, 2005 11:50 AM
> To: Lucene Users List
> Subject: RE: Search Performance
> 
> IndexSearchers are thread safe, so you can use the same object on multiple
> requests.  If the index is static and not constantly updating, just keep one
> IndexSearcher for the life of the app.  If the index changes and you need
> that instantly reflected in the results, you need to check if the index has
> changed, if it has create a new cached IndexSearcher.  To check for changes
> use you'll need to monitor the version number of the index obtained via
> 
> IndexReader.getCurrentVersion(Index Name)
> 
> David
> 
> -----Original Message-----
> From: Stefan Groschupf [mailto:sg@media-style.com]
> Sent: 18 February 2005 16:15
> To: Lucene Users List
> Subject: Re: Search Performance
> 
> Try a singleton pattern or an static field.
> 
> Stefan
> 
> Michael Celona wrote:
> 
> >I am creating new IndexSearchers... how do I cache my IndexSearcher...
> >
> >Michael
> >
> >-----Original Message-----
> >From: David Townsend [mailto:david.townsend@magus.co.uk]
> >Sent: Friday, February 18, 2005 11:00 AM
> >To: Lucene Users List
> >Subject: RE: Search Performance
> >
> >Are you creating new IndexSearchers or IndexReaders on each search?
> Caching
> >your IndexSearchers has a dramatic effect on speed.
> >
> >David Townsend
> >
> >-----Original Message-----
> >From: Michael Celona [mailto:mcelona@criticalmention.com]
> >Sent: 18 February 2005 15:55
> >To: Lucene Users List
> >Subject: Search Performance
> >
> >
> >What is single handedly the best way to improve search performance?  I have
> >an index in the 2G range stored on the local file system of the searcher.
> >Under a load test of 5 simultaneous users my average search time is ~4700
> >ms.  Under a load test of 10 simultaneous users my average search time is
> >~10000 ms.    I have given the JVM 2G of memory and am a using a dual 3GHz
> >Zeons.  Any ideas?
> >
> >
> >
> >Michael
> >
> >
> >---------------------------------------------------------------------
> >To unsubscribe, e-mail: lucene-user-unsubscribe@jakarta.apache.org
> >For additional commands, e-mail: lucene-user-help@jakarta.apache.org
> >
> >
> >
> >---------------------------------------------------------------------
> >To unsubscribe, e-mail: lucene-user-unsubscribe@jakarta.apache.org
> >For additional commands, e-mail: lucene-user-help@jakarta.apache.org
> >
> >
> >
> >
> 
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: lucene-user-unsubscribe@jakarta.apache.org
> For additional commands, e-mail: lucene-user-help@jakarta.apache.org
> 
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: lucene-user-unsubscribe@jakarta.apache.org
> For additional commands, e-mail: lucene-user-help@jakarta.apache.org
> 
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: lucene-user-unsubscribe@jakarta.apache.org
> For additional commands, e-mail: lucene-user-help@jakarta.apache.org
> 
>

---------------------------------------------------------------------
To unsubscribe, e-mail: lucene-user-unsubscribe@jakarta.apache.org
For additional commands, e-mail: lucene-user-help@jakarta.apache.org


Mime
View raw message