lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From ST ST <stst2...@gmail.com>
Subject Re: Query Timings increase after system is idle
Date Tue, 01 Jun 2010 14:02:57 GMT
Thanks for the response, Mike.

In our solution, the entire index is memory mapped.


On Thu, May 20, 2010 at 2:05 PM, Mike Klaas <mike.klaas@gmail.com> wrote:

> Solr and Lucene rely on the OS disk cache for speedy queries.  If they
> are left idle for hours while the disk is used for other purposes, the
> OS will page out some of the lucene index (and it will have to be
> paged back in the next time a query is received).
>
> The best way to prevent this from occurring is to set up a periodic
> automatic query using crontab that keeps the system warm.
>
> cheers,
> -Mike
>
> On Wed, May 19, 2010 at 2:19 PM, ST ST <stst2020@gmail.com> wrote:
> > Folks,
> >
> > We have a problem in our environment where after a system is idle the
> query
> > time goes up from a few 100ms to 4+ seconds after 9 hours of idle time on
> > the system.
> >
> > System Details:
> >  - Solr 1.4 with Lucene 2.9
> >  - 10 Million Index.
> >  - Use MMAP for mapping the index files in memory
> >
> > Test Details:
> > -  8 hour performance run with ingestion (@ 8 docs/sec) , query rate - 3
> > Queries per sec.
> > -  Commit is per hour.
> >
> > Issue:
> > - After 9 hours of idle time (ie no queries, no ingestion ) every query
> > takes 4+ seconds, subsequent queries are fast.
> >
> > I have a few specific questions:
> > A. Does Lucene/Solr have internal caches which may be flushed out of
> memory
> > when the system is idle ?
> > B. What operations are done on a per term basis (example: build doc lists
> )
> > for first time queries.
> > C. Any pointers to what else may be an issue here.
> >
> > Really appreciate any help you can provide.
> >
> > ST
> >
> >
> >
> >
> >
> >
> >
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
> For additional commands, e-mail: dev-help@lucene.apache.org
>
>

Mime
View raw message