lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Paul Elschot <paul.elsc...@xs4all.nl>
Subject Re: TermScorer default buffer size
Date Wed, 07 Jan 2009 13:24:15 GMT
On Wednesday 07 January 2009 07:25:17 John Wang wrote:
> Hi:
> 
>    The default buffer size (for docid,score etc) is 32 in TermScorer.
> 
>     We have a large index with some terms to have very dense doc sets. By
> increasing the buffer size we see very dramatic performance improvements.
> 
>     With our index (may not be typical), here are some numbers with buffer
> size w.r.t. performance in our query (a large OR query):
> 
>     Buffer-size  improvement
> 2042 -       22.0 %
> 4084 -       39.1 %
> 8172 -       51.1 %
> 
>     I understand this may not be suitable for every application, so do you
> think it makes sense to make this buffer size configurable?
> 

Ideally the TermScorer buffer size could be set to a size depending on
the query structure, but there is no facility for this yet.
For OR queries larger buffers help, but not for AND queries.
See also LUCENE-430 on reducing buffer sizes for the underlying
TermDocs for very sparse doc sets.

Regards,
Paul Elschot

Mime
  • Unnamed multipart/alternative (inline, 7-Bit, 0 bytes)
View raw message