lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Victor Lee <victor_email...@yahoo.com>
Subject Re: Is It a Good Idea to Save Frequently Search Results in Database to Make It Faster?
Date Fri, 25 Nov 2005 21:31:03 GMT
Actually, I am planning to use the exact kinds of queries Nutch is using, except that I am
using memoryindex.  So I am match a bunch of queries to a single page, so it's almost exactly
like Nutch except doing it in the opposite way.  So I think that it's more intense than TermQuery.
 
 
 What do you think?

Erik Hatcher <erik@ehatchersolutions.com> wrote: Speculating about a possible future
performance issue is like  
predicting the weather.

You need to measure and find out given your data and how you want to  
architect your index, and of course it is hardware and software (in  
how you use Lucene) dependent too.  Doing a search with Lucene is  
different than a SQL SELECT... you don't actually get the data back  
immediately from a search, so you'd fetch the actual documents, which  
very likely is only the top 20 or so at a time.

If I was a betting man, I'd put my money on Lucene beating MySQL in  
the TermQuery scenario you described (e=hello) ;)   But you'd be wise  
to step out of design mode and get some real-world tests going.  And  
even if there is a performance difference, we're talking milliseconds  
most likely.

 Erik


On 24 Nov 2005, at 22:48, Victor Lee wrote:

> No, I am in the middle of development, actually more like design  
> phase.  So I don't know the speed of my searches.  The final  
> product will have heavy load on SELECT query if using database with  
> possibility of joining 2 tables.  I don't worry about speed of  
> searches during light traffic.  I am worry about what happens  
> during heavy load.
>
>  For a simple example, if I am doing SELECT a, b, c, d FROM mytable  
> WHERE e = hello in a mysql database.  And if I am doing the same  
> thing in Lucene by search "hello" in index and make it return  
> a,b,c,d.  And I do all these continuously to simulate heavy load.   
> Which one is more likely to have more queries per second?  I think  
> it would be mysql based on what I read on their benchmarks on the  
> web.  what do you think?
>
> Erik Hatcher  wrote: How fast are your  
> searches currently?  With just a pure
> searcher.search(Query) using a basic query type like TermQuery, I
> very seriously doubt you'd beat MySQL performance.  What kind of
> Query are you using for your searches?
>
>  Erik
>
>
> On 24 Nov 2005, at 17:54, Victor Lee wrote:
>
>> Sorry, actually I meant all search results, not just frequent
>> results.  And there is only one search term per search, it's the
>> stuff that belongs to the search terms change often.
>>
>> Victor Lee  wrote:  Hi,
>>  I use Lucene to index stuff that are changed very often but don't
>> need to be real-time to searchers. e.g. the search result can be
>> changed couple times per minute, but I only need to show the change
>> every 5 minutes or so. Is it a good idea to save the search result
>> to a database like mysql db, so that searches will load from db
>> directly rather than doing the same search over and over again?
>> Then after 5 minutes, the system will do the actual search again
>> and then save search result to the db for next 3-5 minutes' searches?
>>
>>  Will it make the server handle heavy load better? Since most of
>> the access goes into the database server rather than the webserver
>> with java in backend. Of course, there will be more to implement
>> and maintain.
>>
>>  Many thanks.
>>
>>
>>
>> __________________________________________________
>> Do You Yahoo!?
>> Tired of spam?  Yahoo! Mail has the best spam protection around
>> http://mail.yahoo.com
>>
>>
>> __________________________________________________
>> Do You Yahoo!?
>> Tired of spam?  Yahoo! Mail has the best spam protection around
>> http://mail.yahoo.com
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org
>
>
>
>
> __________________________________________________
> Do You Yahoo!?
> Tired of spam?  Yahoo! Mail has the best spam protection around
> http://mail.yahoo.com


---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org




		
---------------------------------
 Yahoo! Music Unlimited - Access over 1 million songs. Try it free.
Mime
  • Unnamed multipart/alternative (inline, 8-Bit, 0 bytes)
View raw message