lucene-solr-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Shawn Heisey <>
Subject Re: Solr crawls during replication
Date Fri, 03 Sep 2010 23:16:49 GMT
  On 9/3/2010 12:37 PM, Jonathan Rochkind wrote:
> Is the OS disk cache something you configure, or something the OS just does automatically
based on available free RAM?  Or does it depend on the exact OS?  Thinking about the OS disk
cache is new to me. Thanks for any tips.

Depends on what you want to configure.  If you're referring to the size, 
I haven't seen any way to change it, but I haven't really tried to 
look.  It's pretty much automatic, with the OS using all free RAM as 
necessary.  If the memory is suddenly required for an application, it 
just discards the least used entries from the cache.

Most operating systems will let you tune how likely it is that the OS 
will swap application memory out to disk so it can have more RAM for 
disk cache.  Depending on how that's set, you might actually find your 
applications being swapped out to make room for the cache.  If that's 
happening, things will REALLY slow down.

Linux distributions usually have a default /proc/sys/vm/swappiness 
setting of 60%, which despite being pretty high, is perfectly fine for 
most workloads when you've got enough memory.  If you are not using up 
most of your system memory with applications, it'll never touch the 
swap.  If you're running very tight, you can to adjust that down, but in 
the end you just need more memory.

When it comes to BSD or other genetic UNIX systems, I'm in unknown 
territory, but as far as I know, they all work about the same.

On Windows, the server products come with the memory settings optimized 
for file serving (disk cache), which is awesome for Solr, unless you 
don't have enough RAM.  The desktop products are optimized for applications.

On most of the free OSes like Linux and the BSDs, the output of the 
'free' command will tell you lots of things.  Important numbers are 
"cached" and the amount of swap currently used.  You can also watch 
'vmstat 5' and see these values (and a few others) change over time.  If 
you're on Windows, open the task manager performance tab, resize it up a 
ways, and run it for a while.


View raw message