lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Yang <teddyyyy...@gmail.com>
Subject lucene (search) performance tuning
Date Mon, 21 May 2012 20:58:58 GMT
I'm trying to make my search faster. right now a query like

name:Joe Moe Pizza   address:77 main street  city:San Francisco

in a index with 20mil such short business descriptions (total size about
3GB) takes about 100--200ms.

I profiled the query, most time is spent in TermScorer.score(), as is shown
by the attached yourkit screenshot.




I tried loading the index onto tmpfs (in-memory block device), and also
tried RAMDirectory, neither helps much.

I am reading http://www.cnlp.org/presentations/slides/AdvancedLuceneEU.pdf
it mentions
Size
– Stopword removal
– Stemming
• Lucene has a number of stemmers available
• Light versus Aggressive
• May prevent fine-grained matches in some cases
– Not a linear factor (usually) due to index compression

so for "stopword removal", I'm already using the standard analyzer, so stop
word removal is already included, right?

also generally any other tricks to try for reducing the search latency?

Thanks!
Yang

Mime
View raw message