lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Stanislav Jordanov <ste...@sirma.bg>
Subject an alternative to optimize?
Date Fri, 01 Dec 2006 13:11:33 GMT
Guys,

I've already asked this question but nobody answered:

Suppose we have a relatively big index which is continuously updated - 
i.e. new docs get added while some of the old docs get deleted.
For pragmatic reasons we have a restriction on maxMergeDocs so that 
segment files don't get enormously big.
Consider now a segment of max size (i.e. containing maxMergeDocs docs 
hence not eligible for a merge)
It is possible that (as time passes) this segment will have more and 
more of its docs deleted.
But as it is not merge-able it will remain the same size and with lots 
of "wholes" in it which is bad for performance.
The only way that I am aware of to correct this problem is to invoke 
index optimization, which has several drawbacks:
1. it takes a while to optimize a big index.
2. the optimization process always produces a index comprising of a 
single (extremely) large segment.
We can live with 1.
But 2 is undesirable.
Is there a way to "optimize" (in terms of purging its deleted docs) an 
index or a single segment
without ending up with a single segment index?

Best,
Stanislav

---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org


Mime
View raw message