lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From eks dev <eks...@yahoo.co.uk>
Subject Re: Optimizing unordered queries
Date Fri, 26 Jun 2009 14:51:40 GMT

You omitNorms(), did you also omitTf()? 

when something like https://issues.apache.org/jira/browse/LUCENE-1345 gets commited, you will
have a posibility to see some benefits (e.g. by packing single postings lists as Filters).
The code there optimises exactly that case as filters contain no Scoring information ...


try:




----- Original Message ----
> From: Nigel <nigelspleen@gmail.com>
> To: java-user@lucene.apache.org
> Sent: Friday, 26 June, 2009 4:11:53
> Subject: Optimizing unordered queries
> 
> I recently posted some questions about performance problems with large
> indexes.  One key thing about our situation is that we don't need sorted
> results (either by relevance or any other key).  I've been looking into our
> memory usage and tracing through some code, which in combination with the
> recent posts about setTermInfosIndexDivisor got me thinking about the best
> way to do a query where the order of results doesn't matter.
> 
> Currently we're (perhaps naively) doing the equivalent of
> query.weight(searcher).scorer(reader).score(collector).  Obviously there's a
> certain amount of unnecessary calculation that results from this if you
> don't care about sorting.  Are there any general recommendations for
> unordered searching?  (We already omit norms.)
> 
> (More details: Of particular interest are things that access the TermInfos,
> since that's the major source of RAM usage: if a smaller number of TermInfos
> were needed then we could perhaps use an aggressive index divisor setting to
> save RAM without a performance penalty.  For example, I was thinking about a
> custom Similarity implementation that skipped the idf calculations, since
> those have to hit the TermInfos.)
> 
> Thanks,
> Chris



      

---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org


Mime
View raw message