lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Michael McCandless <luc...@mikemccandless.com>
Subject Re: Searching while optimizing
Date Fri, 27 Nov 2009 21:47:24 GMT
Phew, thanks for testing!  It's all explainable...

When you have a reader open, it prevents the segments it had opened
from being deleted.

When you close that reader, the segments could be deleted, however,
that won't happen until the writer next tries to delete, which it does
only periodically (eg, on flushing a new segment, committing a new
merge, etc.).

Could you try closing your reader, then calling writer.commit() (which
is a no-op, since you had already committed, but it may tickle the
writer into attempting the deletions), and see if that frees up disk
space w/o closing?

Mike

On Fri, Nov 27, 2009 at 4:12 PM, vsevel <v.sevel@lombardodier.com> wrote:
> I am starting my tests with an unoptimized 40Mb index. I have 3 test cases:
> 1) open a writer, optimize, commit, close
> 2) open a writer, open a reader from the writer, optimize, commit, close
> 3) same as 2) except the reader is opened while the optimize is done in a
> different thread
>
> During all the tests, I monitor the size of the index on the disk. The
> results are:
> 1) initial=41Mb, before end of optimize=122Mb, after end of optimize=81Mb,
> after commit=40Mb,                            after writer close=40Mb
> 2) initial=41Mb, before end of optimize=122Mb, after end of optimize=104Mb,
> after commit=104Mb, after reader close=104Mb, after writer close=40Mb
> 3) initial=41Mb, before end of optimize=145Mb, after end of optimize=127Mb,
> after commit=103Mb, after reader close=103Mb, after writer close=40Mb
>
> From your different posts I assumed that a commit would have the same effect
> as a close as far as reclaiming disk space is concerned. however test cases
> 2 and 3 show that whether the reader is opened before or during the optimize
> we end up after commit with an index that is 2.5 times the nominal size.
> closing the reader does not change anything. only a close can get us the
> index back to nominal.
>
> What is the reason why the commit nor closing the reader can get us back to
> nominal?
> Do you recommend closing and recreating a new writer after an optimize?
>
> thanks
> vincent
>
>
> Michael McCandless-2 wrote:
>>
>> OK, I'll add that to the javadocs; thanks.
>>
>> But the fact that you weren't closing the old readers was probably
>> also tying up lots of disk space...
>>
>> Mike
>>
>
> --
> View this message in context: http://old.nabble.com/Searching-while-optimizing-tp26485138p26545384.html
> Sent from the Lucene - Java Users mailing list archive at Nabble.com.
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org
>
>

---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org


Mime
View raw message