lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Otis Gospodnetic <>
Subject Re: possible to read index into memory?
Date Wed, 13 Aug 2008 03:08:59 GMT
Another very simple alternative to using RAMDirectory is the use of RAM FS:

Sematext -- -- Lucene - Solr - Nutch

----- Original Message ----
> From: Chris Hostetter <>
> To:
> Sent: Tuesday, August 12, 2008 10:55:28 PM
> Subject: Re: possible to read index into memory?
> : On one index, I am seeing no speed change when flipping between
> : RAMDirectory IndexSearcher and file system version.
> that is probably because even if you just use an FSDirectory, your OS will 
> cache the disk "pages" in RAM for you -- all using a RAMDirectory does for 
> you is garuntee that the entire index is copied into the heap you allocate 
> for your JVM.  If you've got 16GB or RAM, and a 5GB index, and you 
> allocated 12GB of RAM to the JVM and read your index into a RAMDirectory, 
> your index will always be in RAM, no matter what other processes do on 
> your machine.
> If instead you only allocate 6GB of RAM to the JVM, and nothing else is 
> using up the rest of your RAM, the OS has plenty to load the whole index 
> into RAM as part of the filesystem cache once you use it -- but if another 
> process comes along and really needs that RAM (or if something reads a lot 
> of other pages of disk) your index might get bumped from the filesystem 
> cache, and the next few reads could be slow.
> : Creating the RAMDirectory from the on-disk index only takes 0.09
> : seconds. It appears it is not loading the data into memory, but maybe
> : just the file names of the index?
> passing an FSDIrectory to the constructor of a RAMDIrectory uses the 
> Directory.copy() method whose source is fairly straight forward and easy 
> to read -- unless your index is ginormous it's not suprising that it's 
> "fast" particularly if it's already in the filesystem cache.
> -Hoss
> ---------------------------------------------------------------------
> To unsubscribe, e-mail:
> For additional commands, e-mail:

To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message