lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Otis Gospodnetic <otis_gospodne...@yahoo.com>
Subject Re: IndexWriter forceOptimize() ?
Date Fri, 12 Jan 2007 05:14:19 GMT
One day I read email in a different order, I miss replies like this.
If optimize(boolean force) looks more attractive than optimizeForce(), that's fine by me.
 I just want to be able to force the cfs index, even if it's already optimized, to expand.
 Getting it to have a single segment is just a nice bonus here for me.

Regarding that while loop.... it looks like iteration is not needed to force reoptimization.
 I've tested it with CFS and non-CFS indices, with optimized and unoptimized indices, with
and without deletions, and after forced optimization I always ended up with a single segment:

        sis = new SegmentInfos(); 
        sis.read(dir);
        System.out.println("SEGS: " + sis.size());

If nobody speaks up until the weekend, I'll add optimize(boolean force).  We can leave optimize()
and make it call optimize(false);

Otis

----- Original Message ----
From: robert engels <rengels@ix.netcom.com>
To: java-dev@lucene.apache.org
Sent: Thursday, January 11, 2007 3:55:29 PM
Subject: Re: IndexWriter forceOptimize() ?

I agree with the boolean addition.

optimize(false) is a request to maybe optimize, optimize(true) always  
should optimize to a single segment

optimize(false) might check some parameter as to the maximum number  
of segments allowed before an actual optimize if performed.


On Jan 11, 2007, at 2:47 PM, Chris Hostetter wrote:

>
> : What do people here think about adding forceOptimize() to  
> IndexWriter?
>
> I like the idea, but i don't have any value add to offer to the  
> discussion
> of wether the implimentation you suggest is "safe" ... in particular i
> notice that the current optimize method is an iterative loop,  
> presumably
> to make surethat mergeSegments gets called as many times as it  
> needs to
> based on segmentInfos.size() .. your version doesn't seem to have  
> that, so
> does that mean your new version wouldn't allways result in a single
> segment?
>
> another suggestin i have is with the API ... instead of calling it
> "forceOptimize" perhaps the current noarg optimize method should be
> deprecated, and replaced with a new optimize(boolean force) where
> force==true means an optimize will be done, and force==false means an
> optimize will be done if the IndexWriter feels it should be  
> done ... this
> would also address my above concern (assuming it's valid)...
>
> @deprecated use optimize(false)
> public synchronized void optimize() throws IOException { optimize 
> (false); }
> public synchronized void optimize(boolean force) throws IOException {
>   flushRamSegments();
>   while (force ||
>          (segmentInfos.size() > 1 ||
>           (segmentInfos.size() == 1 &&
>            (SegmentReader.hasDeletions(segmentInfos.info(0)) ||
>             segmentInfos.info(0).dir != directory ||
>             (useCompoundFile &&
>              (!SegmentReader.usesCompoundFile(segmentInfos.info(0)) ||
>                SegmentReader.hasSeparateNorms(segmentInfos.info 
> (0)))))))) {
>     int minSegment = segmentInfos.size() - mergeFactor;
>     mergeSegments(segmentInfos, minSegment < 0 ? 0 : minSegment,  
> segmentInfos.size());
>   }
> }
>
>
> -Hoss
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-dev-help@lucene.apache.org
>


---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org





---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org


Mime
View raw message