lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Uwe Schindler <...@thetaphi.de>
Subject RE: Lucene slow performance
Date Sat, 16 Mar 2013 05:16:45 GMT
Please forceMerge only one time not every time (only to clean up your index)! If you are doing
a reindex already, just fix your close logic as discussed before. 



Scott Smith <ssmith@mainstreamdata.com> schrieb:

>Unfortunately, this is a production system which I can't touch (though
>I was able to get a full reindex scheduled for tomorrow morning).  
>
>Are you suggesting that I do:
>
>writer.forceMerge(1);
>writer.close();
>
>instead of just doing the close()?
>
>-----Original Message-----
>From: Simon Willnauer [mailto:simon.willnauer@gmail.com] 
>Sent: Friday, March 15, 2013 5:08 PM
>To: java-user@lucene.apache.org
>Subject: Re: Lucene slow performance
>
>On Sat, Mar 16, 2013 at 12:02 AM, Scott Smith
><ssmith@mainstreamdata.com> wrote:
>> " Do you always close IndexWriter after adding few documents and when
>closing, disable "wait for merge"? In that case, all merges are
>interrupted and the merge policy never has a chance to merge at all
>(because you are opening and closing IndexWriter all the time with
>cancelling all merges)?"
>>
>> Frankly I don't quite understand what this means.  When I "close" the
>indexwriter, I simply call close().  Is that the wrong thing?
>that should be fine...
>
>this sounds very odd though, do you see file that get actually removed
>/ merged if you call IndexWriter#forceMerge(1)
>
>simon
>>
>> Thanks
>>
>> Scott
>>
>> -----Original Message-----
>> From: Uwe Schindler [mailto:uwe@thetaphi.de]
>> Sent: Friday, March 15, 2013 4:49 PM
>> To: java-user@lucene.apache.org
>> Subject: RE: Lucene slow performance
>>
>> Hi,
>>
>> with standard configuartion, this cannot happen. What merge policy do
>you use? This looks to me like a misconfigured merge policy or using
>the NoMergePolicy. With 3,000 segments, it will be slow, the question
>is, why do you get those?
>>
>> Another thing could be: Do you always close IndexWriter after adding
>few documents and when closing, disable "wait for merge"? In that case,
>all merges are interrupted and the merge policy never has a chance to
>merge at all (because you are opening and closing IndexWriter all the
>time with cancelling all merges)?
>>
>> Uwe
>>
>> -----
>> Uwe Schindler
>> H.-H.-Meier-Allee 63, D-28213 Bremen
>> http://www.thetaphi.de
>> eMail: uwe@thetaphi.de
>>
>>> -----Original Message-----
>>> From: Scott Smith [mailto:ssmith@mainstreamdata.com]
>>> Sent: Friday, March 15, 2013 11:15 PM
>>> To: java-user@lucene.apache.org
>>> Subject: Lucene slow performance
>>>
>>> We have a system that is using lucene and the searches are very
>slow.
>>> The number of documents is fairly small (less than 30,000) and each 
>>> document is typically only 2 to 10 kilo-characters.  Yet, searches
>are taking 15-16 seconds.
>>>
>>> One of the things I noticed was that the index directory has several
>
>>> thousand
>>> (3000+) .cfs files.  We do optimize the index once per day.  This is
>
>>> a system that probably gets several thousand document deletes and 
>>> additions per day (spread out across the day).
>>>
>>> Any thoughts.  We didn't really notice this until we went to 4.x.
>>>
>>> Scott
>>>
>>
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
>> For additional commands, e-mail: java-user-help@lucene.apache.org
>>
>
>---------------------------------------------------------------------
>To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
>For additional commands, e-mail: java-user-help@lucene.apache.org
>
>
>---------------------------------------------------------------------
>To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
>For additional commands, e-mail: java-user-help@lucene.apache.org

--
Uwe Schindler
H.-H.-Meier-Allee 63, 28213 Bremen
http://www.thetaphi.de
Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message