lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ian Lea <ian....@gmail.com>
Subject Re: Large indexes
Date Fri, 08 Jul 2011 14:50:04 GMT
There are lots of general tips at
http://wiki.apache.org/lucene-java/ImproveSearchingSpeed.

What version of lucene?  Recent releases should be faster. Have you
tried with one big index? If everything is running on the same server
that may well be faster.

Even on single indexes, response of a few seconds sounds high. Are the
searches complex?  Are you sorting?  Reading lots of hits for each
search?


--
Ian.

On Fri, Jul 8, 2011 at 3:39 PM, Chris Bamford
<chris.bamford@talktalk.net> wrote:
>
>  Hi
>
>
> I was wondering how to improve search performance over a set of indexes like this:
>
>  27G    K1-1/index
>  19G    K1-2/index
>  24G    K1-3/index
>  15G    K1-4/index
>  19G    K1-5/index
>  31G    K2-1/index
>  16G    K2-2/index
>  8.1G   K2-3/index
>  12G    K2-4/index
>  15G    K2-5/index
>
> In total it is ~192Gb.  All have the same schema, with data partitioned by my application
into separate
> indexes (rather than just 1 big one).
> I have tried searching with both a MultiSearcher and a ParallelMultiSearcher (with lots
of memory), but it
> is way too slow (sometimes hangs, otherwise it takes tens of minutes).  Do you have
any tips for improving speed?
> I have reduced the stored data as much as possible (without using compression) and use
a FieldSelector when querying
> but neither seem to help..
>
> Is this the right approach?  If I query them one by one I response time is reasonable
(a few seconds each), but I was
> kind of hoping to treat them as one large index if possible.
>
> Any thoughts welcomed.
>
> Thanks,
>
>  - Chris
>
>

---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org


Mime
View raw message