lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Michael McCandless <luc...@mikemccandless.com>
Subject Re: Explanation on RAMBufferSizeMB
Date Fri, 21 Mar 2008 10:24:34 GMT

Shai Erera wrote:
> Besides the content field, everything is stored, so that may  
> explain the
> large CFS files.

You could run w/o CFS turned on and then look at the size of fdt/fdx  
to see if this explains the size.

> Regarding the RAM-usage performance, I tried setting to 128, 256  
> and 512,
> all gave the same time measurements (give or take ~5%) as the MBD  
> (set to
> 10,000) run. I think it needs further investigation. Was it tested  
> before? I
> mean - has someone tried to set RAM to 2GB for example and noticed  
> a major
> performance improvement (as I'd expect)?

Yes, in LUCENE-843 Doron did some tests and found similar tail-off, I  
think above 32 or 64 MB RAM buffer. I also saw the same thing in my  
tests.

However one thing to remember is: a larger RAM buffer will result in  
less merging.  But, with CMS and a machine with good enough  
concurrency, net run time may be unaffected by this benefit even  
though net resource consumption is.  This makes it hard to test  
speedups in merging :)  Does your test do any merging at all?

Mike

---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org


Mime
View raw message