lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Michael McCandless <luc...@mikemccandless.com>
Subject Re: caching an indexreader
Date Fri, 19 Jun 2009 19:11:40 GMT
On Fri, Jun 19, 2009 at 2:40 PM, Scott Smith<ssmith@mainstreamdata.com> wrote:
> In my environment, one of the concerns is that new documents are
> constantly being added (and some documents may be deleted).  This means
> that when a user does a search and pages through results, it is possible
> that there are new items coming in which affect the search-thus changing
> where items are in relation to the pages displayed to the user.

This is a very real problem that most search apps simply ignore.

> It seems to me that one solution would be to cache the Searcher for the
> duration of the user's search session so that the user's view of the
> available documents doesn't change.  As I understand it, the user won't
> see any changes to the index until a new Searcher is created.  However,
> I'm very sensitive to the amount of session context memory that caching
> the searcher might take up.  How much memory will caching the searcher
> cost?  Are there other tradeoff's I need to consider?

Caching the reader & reader affinity (re-using the same reader when
user pages) is the right approach.

Prior to 2.9 (not yet released) the RAM requirements of multiple
readers were high, if you sort by field or use the FieldCache
directly, because a new FieldCache entry is created for the entire
index, per reader.  It was also slow to reopen for this reason (on a
large index).

With 2.9, the FieldCache entries are stored per segment, so that a
newly opened reader only adds entries for new segments in the index.
Often this is a tiny amount of added RAM, but if a big merge has just
finished it could be alot of RAM.

Mike

---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org


Mime
View raw message