lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Suprakash <suprakash_...@yahoo.com>
Subject weird mergeFactor behavior in v1.4.3
Date Fri, 04 May 2007 20:20:04 GMT

Hi All, 
I'm running Lucene 1.4.3 and have noticed that the mergeFactor setting is
not working as I would expect. To provide a little background - this all
started as an attempt to reduce the number of segments in the index
directory. After several months of use in a high update environment, I end
up with over a thousand (mostly 1KB) files in the index directory and search
performance takes a significant hit. My current settings are mergeFactor=750
& minMergeDocs=750 (maxMergeDocs=default).

I was first considering some kind of scheduled optimization solution, but
experts advise against it:

http://www.gossamer-threads.com/lists/lucene/java-dev/47895?search_string=optimize;#47895

So I started going down the path of lowering the mergeFactor. Here is what
I'm seeing - with mergeFactor=10 and a minMergeDocs=10, as I add 1000 docs
to the index (in one go) 10 segments of 10 docs get created and then rolled
up into 1 segment of 1000 docs. So far so good. However if I add 900 docs to
the index and then add 100 docs in a second pass (closing the IndexWriter
between the two runs), I get 10 segments on disk but they are not merged.
Also, if I add 10 docs with mergeFactor=1, I get 10 segments. 

Is this expected? I would have thought that a) segments are merged as they
are created regardless of timing b) a mergeFactor=1 would always keep the
index optimized. Am I running into a bug in version 1.4.3? Any ideas for how
to get around this? Upgrading to the latest version is not an option in my
case (I'm at the very end of a release cycle).

Thanks very much in advance!
Das
-- 
View this message in context: http://www.nabble.com/weird-mergeFactor-behavior-in-v1.4.3-tf3693916.html#a10329266
Sent from the Lucene - Java Users mailing list archive at Nabble.com.


---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org


Mime
View raw message