lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Erick Erickson <erickerick...@gmail.com>
Subject Re: multiple small indexes or one big index?
Date Wed, 01 Jun 2011 21:26:18 GMT
I'd start by putting them all in one index. There's no penalty
in Lucene for having empty fields in a document, unlike an
RDBMS.

Alternately, if you're opening then closing searchers each
time, that's very expensive. Could you open the searchers
once and keep them open (all 90 of them)? That alone might
do the trick and be less of a change to your program. You
could also fire multiple threads at the searches, but check if
you're CPU bound first (if you are, multiple threads won't
help much/at all).

You haven't said how big these indexes are nor how many
documents you're talking about here, so this advice is suspect.

Do look at putting it all in one index though, let us know if you
have some data indicating how big stuff is/would be.

Best
Erick

On Wed, Jun 1, 2011 at 4:35 PM, Alexander Rosemann
<alexander.rosemann@gmail.com> wrote:
> Hi all, I was wondering whether you could give me some advice on how to
> improve my search performance.
>
> I have 90 lucene indexes, each having different fields (~5 per Document).
> When I search, I always have to go through all indexes to build my result
> set. Searching one index takes approx. 100ms, thus searching all indexes
> takes 9s in total.
>
> How can I reduce the time it needs to search?
>
> I decided to create this many indexes because putting all data in one index
> would mean that a document would have ~400 fields, with most of them left
> empty. Is that ok? Would a single index be faster compared to multiple small
> ones?
>
> Any pointers are much appreciated.
>
> Regards,
> Alex
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org
>
>

---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org


Mime
View raw message