lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Doug Cutting <DCutt...@grandcentral.com>
Subject RE: CachingDirectory contribution
Date Mon, 08 Oct 2001 16:23:12 GMT
> From: Maik Schreiber [mailto:bZ@iq-computing.de]
> 
> A while back I wrote a CachingDirectory implementation for 
> Lucene which
> allows for caching an index on a local machine other than the "root"
> machine. This can be very useful for handling heavy load 
> (such as David
> Snyder's 13 GB index :-))
> 
> I'd really love to see it included in the Lucene package (I'm 
> okay with
> it being put under the Apache license). If there's some 
> interest I could
> provide the sources to the abstract CachingDirectory as well as actual
> FSCachingDirectory and RAMCachingDirectory implementations.

Please post them.  If folks find them useful, they can be added to Lucene.

There are lots of places to cache.  The operating system usually makes a
valiant effort to cache files on the local FS using available RAM.  NFS
mounted files are also usually cached in RAM, and there are even NFS client
implementation which cache entire files on the local FS, so that if they
fall out of the RAM cache then a network read is not required.  Explicitly
caching multi-gigabyte files in the Java heap is probably not practical.
But if NFS or the local FS do not cache things adequately, there might be a
place for Java-based caches that use either RAM or the local FS.  I'm not
sure what they are, but that doesn't mean they don't exist!  Benchmarking is
always the best way to determine these things, and if folks find that a
cache helps, let's add it.

Doug

Mime
View raw message