lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Otis Gospodnetic <otis_gospodne...@yahoo.com>
Subject Re: Query Timings increase after system is idle
Date Tue, 01 Jun 2010 04:15:52 GMT
Hi,

You are most likely seeing the effects of your previously cached (by the OS) index data getting
paged out of the cache by other disk and process data.  The OS is doing the right thing -
if nobody is using index data, why keep it in memory when that memory could be used for other
things.

Is this happening under Linux?  If so, this "swappiness" can be controlled: http://kerneltrap.org/node/3000

 Otis
----
Sematext :: http://sematext.com/ :: Solr - Lucene - Nutch
Lucene ecosystem search :: http://search-lucene.com/


>
>From: ST ST <stst2020@gmail.com>
>To: dev@lucene.apache.org
>Sent: Wed, May 19, 2010 5:19:09 PM
>Subject: Query Timings increase after system is idle
>
>Folks,
>
>
>We have a problem in our environment where after a system is idle the query time goes
up from a few 100ms to 4+ seconds after 9 hours of idle time on the system.
>
>System Details:
>> - Solr 1.4 with Lucene 2.9
> - 10 Million Index.
>> - Use MMAP for mapping the index files in memory
>
>Test Details:
>-  8 hour performance run with ingestion (@ 8 docs/sec) , query rate - 3 Queries per sec.
>-  Commit is per hour.
>
>Issue:
>- After 9 hours of idle time (ie no queries, no ingestion ) every query takes 4+ seconds,
subsequent queries are fast.
>
>I have a few specific questions:
>A. Does Lucene/Solr have internal caches which may be flushed out of memory when the system
is idle ?
>B. What operations are done on a per term basis (example: build doc lists ) for first
time queries.
>>
>C. Any pointers to what else may be an issue here.
>
>Really appreciate any help you can provide.
>
>ST
>
>
>
>
>
>
>
Mime
View raw message