lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From bugzi...@apache.org
Subject DO NOT REPLY [Bug 28183] - [Patch] replace DocumentWriter with InvertedDocument for performance
Date Wed, 07 Apr 2004 03:35:59 GMT
DO NOT REPLY TO THIS EMAIL, BUT PLEASE POST YOUR BUG 
RELATED COMMENTS THROUGH THE WEB INTERFACE AVAILABLE AT
<http://issues.apache.org/bugzilla/show_bug.cgi?id=28183>.
ANY REPLY MADE TO THIS MESSAGE WILL NOT BE COLLECTED AND 
INSERTED IN THE BUG DATABASE.

http://issues.apache.org/bugzilla/show_bug.cgi?id=28183

[Patch] replace DocumentWriter with InvertedDocument for performance





------- Additional Comments From brian-apache@slesinsky.org  2004-04-07 03:35 -------
Yes, the patch is not the easiest to read.  Sorry about that!  I thought I'd send it in to
get the 
conversation started.  Unfortunately I needed to do a bit of refactoring before starting (to
make it 
possible to subclass SegmentInfo), so it wouldn't be a very clean patch anyway.

As far as memory usage goes, in my particular case I don't think it matters.   I'm running
java with 
256M of memory on a machine with  768M, and java isn't making much use of the memory it has.
 
(Reducing Java's memory to 128M results in only a 10% slowdown due to increased garbage collection.)
 
Raising minMergeDocuments beyond about 500 seems to result in no performance improvement,
either 
with or without the patch.  The task is CPU-bound with extra memory and I/O bandwidth available,
so it 
looks like trading memory for performance makes sense even if the memory usage is higher.

However, perhaps this isn't true for other machines and/or other datasets?  Do you have an
example of 
a workload where memory is the bottleneck?  Or perhaps I'm missing something about how to
tune 
Lucene?

---------------------------------------------------------------------
To unsubscribe, e-mail: lucene-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: lucene-dev-help@jakarta.apache.org


Mime
View raw message