There are lots of general tips at
http://wiki.apache.org/lucene-java/ImproveSearchingSpeed.
What version of lucene? Recent releases should be faster. Have you
tried with one big index? If everything is running on the same server
that may well be faster.
Even on single indexes, response of a few seconds sounds high. Are the
searches complex? Are you sorting? Reading lots of hits for each
search?
--
Ian.
On Fri, Jul 8, 2011 at 3:39 PM, Chris Bamford
<chris.bamford@talktalk.net> wrote:
>
> Hi
>
>
> I was wondering how to improve search performance over a set of indexes like this:
>
> 27G K1-1/index
> 19G K1-2/index
> 24G K1-3/index
> 15G K1-4/index
> 19G K1-5/index
> 31G K2-1/index
> 16G K2-2/index
> 8.1G K2-3/index
> 12G K2-4/index
> 15G K2-5/index
>
> In total it is ~192Gb. All have the same schema, with data partitioned by my application
into separate
> indexes (rather than just 1 big one).
> I have tried searching with both a MultiSearcher and a ParallelMultiSearcher (with lots
of memory), but it
> is way too slow (sometimes hangs, otherwise it takes tens of minutes). Do you have
any tips for improving speed?
> I have reduced the stored data as much as possible (without using compression) and use
a FieldSelector when querying
> but neither seem to help..
>
> Is this the right approach? If I query them one by one I response time is reasonable
(a few seconds each), but I was
> kind of hoping to treat them as one large index if possible.
>
> Any thoughts welcomed.
>
> Thanks,
>
> - Chris
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org
|