lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Otis Gospodnetic <otis_gospodne...@yahoo.com>
Subject Setting swappiness
Date Wed, 24 Jun 2009 14:02:05 GMT

Stealing this thread/idea, but changing subject, so we can branch and I don't look like a
thread thief.


I never played with /proc/sys/vm/swappiness, but I wonder if there are points in the lifetime
of an index where this number should be changed.  For example, does it make sense to in/decrease
that number once we know the index is going to be read-only for a while?  Does it make sense
to in/decrease it during merges or optimizations?

Thanks,
Otis
--
Sematext -- http://sematext.com/ -- Lucene - Solr - Nutch



----- Original Message ----
> From: Michael McCandless <lucene@mikemccandless.com>
> To: java-user@lucene.apache.org
> Sent: Wednesday, June 24, 2009 5:06:25 AM
> Subject: Re: Analyzing performance and memory consumption for boolean queries
> 
> Is it possible the occasional large merge is clearing out the IO cache
> (thus "unwarming" your searcher)?  (Though since you're rsync'ing your
> updates in, it sounds like a separate machine is building the index).
> 
> Or... linux will happily swap out a process's core in favor of IO
> cache (though I'd expect this effect to be much less spikey).  You can
> tune "swappiness" to have it not do that:
> 
>     http://kerneltrap.org/node/3000
> 
> Maybe Lucene's norms/deleted docs/field cache were getting swapped out?
> 
> Lucene's postings reside entirely on disk (ie, Lucene doesn't cache
> those in RAM; we rely on the OS's IO cache).  Lucene does a linear
> scan through the terms in the query... Linux will readahead, though,
> if things are fragmented this could mean lots of seeking.

---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org


Mime
View raw message