lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Erik Hatcher <e...@ehatchersolutions.com>
Subject Re: Search Performance Problem 16 sec for 250K docs
Date Sun, 20 Aug 2006 12:52:06 GMT
This is why a warming strategy like Solr takes is very valuable.  The  
searchable index is always serving up requests as fast as Lucene  
works, which is achieved by warming a new IndexSearcher with searches/ 
sorts/filter creating/etc before it is swapped into use.

	Erik


On Aug 20, 2006, at 5:35 AM, M A wrote:

> Ok I get your point, this still however means the first search on  
> the new
> searcher will take a huge amount of time .. given that this is  
> happening now
> ..
>
> i.e. new search -> new query -> get hits ->20+ secs ..  this  
> happens every 5
> mins or so ..
>
> although subsequent searches may be quicker ..
>
> Am i to assume for a first search the amount of  time is ok -> ..  
> seems like
> a long time to me ..?
>
> The other thing is the sorting is fixed .. it never changes .. it  
> is always
> sorted by the same field ..
>
> i just built the entire index and it still takes ages .,..
>
>
>
>
>
>
>
>
> On 8/20/06, Chris Hostetter <hossman_lucene@fucit.org> wrote:
>>
>>
>> : This is because the index is updated every 5 mins or so, due to the
>> incoming
>> : feed of stories ..
>> :
>> : When you say iteration, i take it you mean, search request, well  
>> for
>> each
>> : search that is conducted I create a new one .. search reader  
>> that is ..
>>
>> yeah ... i ment iteration of your test.  don't do that.
>>
>> if the index is updated every 5 minutes, then open a new searcher  
>> every 5
>> minutes -- and reuse it for theentire 5 minutes.  if it's updated
>> "sparadically throughout the day" then open a search, and keep  
>> using it
>> untill the index is udated, then open a new one.
>>
>> reusing an indexsearcher as long as possible is one of biggest  
>> factors of
>> Lucene applications.
>>
>> :
>> :
>> :
>> : On 8/19/06, Chris Hostetter <hossman_lucene@fucit.org> wrote:
>> : >
>> : >
>> : > :     hits = searcher.search(query, new Sort("sid", true));
>> : >
>> : > you don't show where searcher is initialized, and you don't  
>> clarify
>> how
>> : > you are timing your multiple iterations -- i'm going to guess  
>> that you
>> are
>> : > opening a new searcher every iteration right?
>> : >
>> : > sorting on a field requires pre-computing an array of  
>> information for
>> that
>> : > field -- this is both time and space expensive, and is cached per
>> : > IndexReader/IndexSearcher -- so if you reuse the same searcher  
>> and
>> time
>> : > multiple iterations you'll find that hte first iteration might be
>> somewhat
>> : > slow, but the rest should be very fast.
>> : >
>> : >
>> : >
>> : > -Hoss
>> : >
>> : >
>> : >  
>> ---------------------------------------------------------------------
>> : > To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
>> : > For additional commands, e-mail: java-user-help@lucene.apache.org
>> : >
>> : >
>> :
>>
>>
>>
>> -Hoss
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
>> For additional commands, e-mail: java-user-help@lucene.apache.org
>>
>>


---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org


Mime
View raw message