lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Kainth, Sachin" <Sachin.Kai...@atkinsglobal.com>
Subject RE: Caching
Date Wed, 14 Feb 2007 14:30:59 GMT
Well,

I have an index with 5.2 million records (each record containing 3
fields) and it sometimes takes about a minute and a half for results to
come back.  I have noticed however, that when I run the same query the
second time the result comes back faster.  I just thought that this was
a bit too slow and that perhaps I should implement a caching strategy. 

-----Original Message-----
From: Erick Erickson [mailto:erickerickson@gmail.com] 
Sent: 14 February 2007 14:11
To: java-user@lucene.apache.org
Subject: Re: Caching

This is really an unanswerable question, since, to steal a phrase, "It
depends" <G>...

Do you have any reason to believe that the current performance is
inadequate for you application? Caching is notoriously difficult to get
right, so I wouldn't go there unless there is a *demonstrated* need. By
demonstrated, I mean that you run some tests against your actual data
with simulated queries and measure. By definition, Lucene's caching is
"any good" if your performance meets your target as is. Only if your
application does NOT meet your targets should you consider an additional
caching layer. And even in this case, I'd spend significant time looking
at exactly *where* in my application things were slow rather than jump
to caching as an answer.

I do not believe that Lucene actually caches the results of queries. I
believe that Lucene builds caches of terms, sorting information, etc.
that are then applied to all subsequent queries, and that these are the
caches that are built at the first query. So, say storing the results of
a query somewhere and returning that stored copy for the *next* query
that is identical is not something I'd expect Lucene to do.

Best
Erick

On 2/14/07, Kainth, Sachin <Sachin.Kainth@atkinsglobal.com> wrote:
>
> Hi all,
>
> I have read that Lucene performs caching of search results so that if 
> you perform the same search in succession the second result should be 
> returned faster.  What I wanted to ask is whether this caching is any 
> good or whether it's a good idea to add some sort of caching layer on 
> top of Lucene to improve on it.
>
> Any comments will be appreciated.
>
> Thanks
>
> Sachin
>
>
>
> This email and any attached files are confidential and copyright 
> protected. If you are not the addressee, any dissemination of this 
> communication is strictly prohibited. Unless otherwise expressly 
> agreed in writing, nothing stated in this communication shall be
legally binding.
>
> The ultimate parent company of the Atkins Group is WS Atkins plc.  
> Registered in England No. 1885586.  Registered Office Woodcote Grove, 
> Ashley Road, Epsom, Surrey KT18 5BW.
>
> Consider the environment. Please don't print this e-mail unless you 
> really need to.
>


This message has been scanned for viruses by MailControl - (see
http://bluepages.wsatkins.co.uk/?4318150)

---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org


Mime
View raw message