lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Otis Gospodnetic <>
Subject Setting swappiness
Date Wed, 24 Jun 2009 14:02:05 GMT

Stealing this thread/idea, but changing subject, so we can branch and I don't look like a
thread thief.

I never played with /proc/sys/vm/swappiness, but I wonder if there are points in the lifetime
of an index where this number should be changed.  For example, does it make sense to in/decrease
that number once we know the index is going to be read-only for a while?  Does it make sense
to in/decrease it during merges or optimizations?

Sematext -- -- Lucene - Solr - Nutch

----- Original Message ----
> From: Michael McCandless <>
> To:
> Sent: Wednesday, June 24, 2009 5:06:25 AM
> Subject: Re: Analyzing performance and memory consumption for boolean queries
> Is it possible the occasional large merge is clearing out the IO cache
> (thus "unwarming" your searcher)?  (Though since you're rsync'ing your
> updates in, it sounds like a separate machine is building the index).
> Or... linux will happily swap out a process's core in favor of IO
> cache (though I'd expect this effect to be much less spikey).  You can
> tune "swappiness" to have it not do that:
> Maybe Lucene's norms/deleted docs/field cache were getting swapped out?
> Lucene's postings reside entirely on disk (ie, Lucene doesn't cache
> those in RAM; we rely on the OS's IO cache).  Lucene does a linear
> scan through the terms in the query... Linux will readahead, though,
> if things are fragmented this could mean lots of seeking.

To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message