lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Eks Dev (JIRA)" <>
Subject [jira] Commented: (LUCENE-893) Increase buffer sizes used during searching
Date Sat, 26 May 2007 09:57:16 GMT


Eks Dev commented on LUCENE-893:

how many documents were in index? 
I assume in case where posting are longer bigger buffer brings more benefit,  especially for
queries that are on dense terms. 

I am bringing this up as optimizing for "general case" is hard/impossible due to too many
scenarios, influences. Having said that:

1. Maybe we constrain things a bit,  e.g number of docs in index set to 10 Mio as a "golden
standard"  target for optimization (high number makes sense, as for smaller indexes it is
already fast enough, 10% faster from stunning fast is just not all that relevant)... maybe
some other ideas what could be considered.
2. Making it configurable, so everybody could tweak it, you never know what HW or file system
one uses...

> Increase buffer sizes used during searching
> -------------------------------------------
>                 Key: LUCENE-893
>                 URL:
>             Project: Lucene - Java
>          Issue Type: Improvement
>          Components: Store
>    Affects Versions: 2.1
>            Reporter: Michael McCandless
> Spinoff of LUCENE-888.
> In LUCENE-888 we increased buffer sizes that impact indexing and found
> substantial (10-18%) overall performance gains.
> It's very likely that we can also gain some performance for searching
> by increasing the read buffers in BufferedIndexInput used by
> searching.
> We need to test performance impact to verify and then pick a good
> overall default buffer size, also being careful not to add too much
> overall HEAP RAM usage because a potentially very large number of
> BufferedIndexInput instances are created during searching
> (# segments X # index files per segment).

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message