lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Bernd Fehling <bernd.fehl...@uni-bielefeld.de>
Subject Re: questions about the index
Date Tue, 03 May 2011 08:52:35 GMT

Well, it is not only with a huge index.
It is only if ReplicationHandler is in use on a master.
If ReplicationHandler is configured to replicateAfter startup it first
sends a commit via IndexWriter to have a "stable" index. The left over
of this operation is the write.lock.
So removing replicateAfter startup removes the write.lock when starting
with an optimized index and replication on a master.

To solve this tiny issue I would recommend to also send an optimize
after sending a commit if the index has state optimize=true.

Bernd


Am 03.05.2011 09:22, schrieb Bernd Fehling:
> Hi Mike,
>
> thanks for the infos.
>
> As far as I know a write.lock is created from an IndexWriter.
> So I have to dig into it why an IndexWriter is created just
> on starting solr with an optimized index.
> The problem, this is only with a huge index.
> And also old parts of the index are not cleaned up.
> May be somewhere around the DeletionPolicy...
>
> Regards, Bernd
>
>
> Am 02.05.2011 17:45, schrieb Michael McCandless:
>> On Mon, May 2, 2011 at 9:17 AM, Bernd Fehling
>> <bernd.fehling@uni-bielefeld.de> wrote:
>>> Dear list,
>>>
>>> some questions about the index.
>>> (questions go to the lucene list because it is more about the index itself)
>>>
>>> First my results from CheckIndex:
>>>
>>> Segments file=segments_l6 numSegments=1 version=FORMAT_3_1 [Lucene 3.1]
>>> Checking only these segments: _79s:
>>> 1 of 1: name=_79s docCount=28146692
>>> compound=false
>>> hasProx=true
>>> numFiles=8
>>> size (MB)=141.658,104
>>> diagnostics = {optimize=true, mergeFactor=2,
>>> os.version=2.6.32.13-0.5-xen,
>>> os=Linux, lucene.version=3.1.0 1085809 - 2011-03-26
>>> 18:06:58,
>>> source=merge, os.arch=amd64, java.version=1.6.0_21,
>>> java.vendor=Sun Microsystems Inc.}
>>> no deletions
>>> test: open reader.........OK
>>> test: fields..............OK [106 fields]
>>> test: field norms.........OK [18 fields]
>>> ....
>>>
>>> Questions:
>>> - the mergeFactor in solrconfig.xml (indexDefaults and mainIndex)
>>> is set to 10. Why is the index reporting mergeFactor=2 ???
>>>
>>> - what does "optimize=true" mean?
>>> - the index is optimized OR
>>> - the index needs to get optimized
>>
>> optimize=true means this segment was produced by a merge trigged by an
>> optimize() call. mergeFactor=2 means that 2 segments were merged to
>> produce this final segment; it's entirely possible that your
>> configured mergeFactor is 10, you called optimize, and the last merge
>> was to merge two segments down to this final one.
>>
>>> - if I start solr with an optimized index I always get a "write.lock" file.
>>> if I use a small index with only a few docs I don't seen this behavior.
>>> any explanation?
>>
>> That seems odd -- write.lock should be created regardless of what the
>> current index looks like, I think.
>>
>> Mike
>>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org
>

-- 
*************************************************************
Bernd Fehling                Universit├Ątsbibliothek Bielefeld
Dipl.-Inform. (FH)                        Universit├Ątsstr. 25
Tel. +49 521 106-4060                   Fax. +49 521 106-4052
bernd.fehling@uni-bielefeld.de                33615 Bielefeld

BASE - Bielefeld Academic Search Engine - www.base-search.net
*************************************************************

---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org


Mime
View raw message