lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Doug Cutting (JIRA)" <>
Subject [jira] Commented: (LUCENE-1035) Optional Buffer Pool to Improve Search Performance
Date Fri, 26 Oct 2007 04:24:50 GMT


Doug Cutting commented on LUCENE-1035:

Were the tests run using the same set of queries they were warmed for?  If so, an interesting
benchmark might be to, e.g., start with 200 queries, then warm things with the first 100 and
use the second for the benchmark.  Ideally you'd start with a log of real queries, but those
are hard to obtain.  Over ten years ago I released a 1M query log from Excite, which I still
see people reference in papers, so it must be out there somewhere.  It would be better than
nothing for these kinds of benchmarks.  Or perhaps we can obtain a copy of the more-recent
AOL query log?  Otherwise you've only demonstrated an improvement when queries are frequently
repeated.  There are better ways to optimize for that, e.g., by caching hit lists, no?

> Optional Buffer Pool to Improve Search Performance
> --------------------------------------------------
>                 Key: LUCENE-1035
>                 URL:
>             Project: Lucene - Java
>          Issue Type: Improvement
>          Components: Store
>            Reporter: Ning Li
>         Attachments: LUCENE-1035.patch
> Index in RAMDirectory provides better performance over that in FSDirectory.
> But many indexes cannot fit in memory or applications cannot afford to
> spend that much memory on index. On the other hand, because of locality,
> a reasonably sized buffer pool may provide good improvement over FSDirectory.
> This issue aims at providing such an optional buffer pool layer. In cases
> where it fits, i.e. a reasonable hit ratio can be achieved, it should provide
> a good improvement over FSDirectory.

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message