lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Scott Smith <ssm...@mainstreamdata.com>
Subject RE: Lucene slow performance
Date Sat, 16 Mar 2013 19:27:48 GMT
Thanks for the help.

The reindex was done this morning and searches now take less than a second.

I will make the change to the code.

Cheers

Scott

-----Original Message-----
From: Uwe Schindler [mailto:uwe@thetaphi.de] 
Sent: Friday, March 15, 2013 11:17 PM
To: java-user@lucene.apache.org
Subject: RE: Lucene slow performance

Please forceMerge only one time not every time (only to clean up your index)! If you are doing
a reindex already, just fix your close logic as discussed before. 



Scott Smith <ssmith@mainstreamdata.com> schrieb:

>Unfortunately, this is a production system which I can't touch (though 
>I was able to get a full reindex scheduled for tomorrow morning).
>
>Are you suggesting that I do:
>
>writer.forceMerge(1);
>writer.close();
>
>instead of just doing the close()?
>
>-----Original Message-----
>From: Simon Willnauer [mailto:simon.willnauer@gmail.com]
>Sent: Friday, March 15, 2013 5:08 PM
>To: java-user@lucene.apache.org
>Subject: Re: Lucene slow performance
>
>On Sat, Mar 16, 2013 at 12:02 AM, Scott Smith 
><ssmith@mainstreamdata.com> wrote:
>> " Do you always close IndexWriter after adding few documents and when
>closing, disable "wait for merge"? In that case, all merges are 
>interrupted and the merge policy never has a chance to merge at all 
>(because you are opening and closing IndexWriter all the time with 
>cancelling all merges)?"
>>
>> Frankly I don't quite understand what this means.  When I "close" the
>indexwriter, I simply call close().  Is that the wrong thing?
>that should be fine...
>
>this sounds very odd though, do you see file that get actually removed 
>/ merged if you call IndexWriter#forceMerge(1)
>
>simon
>>
>> Thanks
>>
>> Scott
>>
>> -----Original Message-----
>> From: Uwe Schindler [mailto:uwe@thetaphi.de]
>> Sent: Friday, March 15, 2013 4:49 PM
>> To: java-user@lucene.apache.org
>> Subject: RE: Lucene slow performance
>>
>> Hi,
>>
>> with standard configuartion, this cannot happen. What merge policy do
>you use? This looks to me like a misconfigured merge policy or using 
>the NoMergePolicy. With 3,000 segments, it will be slow, the question 
>is, why do you get those?
>>
>> Another thing could be: Do you always close IndexWriter after adding
>few documents and when closing, disable "wait for merge"? In that case, 
>all merges are interrupted and the merge policy never has a chance to 
>merge at all (because you are opening and closing IndexWriter all the 
>time with cancelling all merges)?
>>
>> Uwe
>>
>> -----
>> Uwe Schindler
>> H.-H.-Meier-Allee 63, D-28213 Bremen
>> http://www.thetaphi.de
>> eMail: uwe@thetaphi.de
>>
>>> -----Original Message-----
>>> From: Scott Smith [mailto:ssmith@mainstreamdata.com]
>>> Sent: Friday, March 15, 2013 11:15 PM
>>> To: java-user@lucene.apache.org
>>> Subject: Lucene slow performance
>>>
>>> We have a system that is using lucene and the searches are very
>slow.
>>> The number of documents is fairly small (less than 30,000) and each 
>>> document is typically only 2 to 10 kilo-characters.  Yet, searches
>are taking 15-16 seconds.
>>>
>>> One of the things I noticed was that the index directory has several
>
>>> thousand
>>> (3000+) .cfs files.  We do optimize the index once per day.  This is
>
>>> a system that probably gets several thousand document deletes and 
>>> additions per day (spread out across the day).
>>>
>>> Any thoughts.  We didn't really notice this until we went to 4.x.
>>>
>>> Scott
>>>
>>
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
>> For additional commands, e-mail: java-user-help@lucene.apache.org
>>
>
>---------------------------------------------------------------------
>To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
>For additional commands, e-mail: java-user-help@lucene.apache.org
>
>
>---------------------------------------------------------------------
>To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
>For additional commands, e-mail: java-user-help@lucene.apache.org

--
Uwe Schindler
H.-H.-Meier-Allee 63, 28213 Bremen
http://www.thetaphi.de
Mime
View raw message