lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Yonik Seeley <ysee...@gmail.com>
Subject Re: major searching performance improvement
Date Thu, 26 May 2005 02:12:57 GMT
Looks like really great stuff Robert!

> 2. I agree that creating NioFSDirectory rather than modifying FSDirectory. I
> originally felt the memory mapped files would be the fastest, but it also
> requires OS calls, the "caching" code is CONSIDERABLY faster, since it does
> not need to do any JNI, or make OS calls.

What do you mean by OS calls required by memory mapped files?

I'm not 100% sure how mmap works in java, but in C the only OS calls
are typically to set up and tear down the mapping.  Reads from the
mmaped region that are already in memory proceed at the same speed as
reads from any other piece of memory.


> The problem with a SoftCache and indices of that
> size, is that the JVM memory consumption would still grow to the limit
> before it discarded anything (which may be ideal in some cases).

Soft caches aren't great, esp with apps that generate a lot of garbage.
What might be better is a multi-level LRU that spills over into soft
references at a certain point.

-Yonik

---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org


Mime
View raw message