lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Erik Hatcher <e...@ehatchersolutions.com>
Subject Re: index file size threshold affecting search performance?
Date Wed, 28 Mar 2007 19:46:19 GMT
I've just built a 9.3G index (admittedly tons of stored data in  
there, 3.3M documents) and performance is amazing (through Solr).

	Erik



On Mar 28, 2007, at 3:11 PM, Erick Erickson wrote:

> This surprises me, I'm currently working with a 4G index, and the
> improvement from when it was an 8G index was only 10% or so.
> And it's plenty speedy.
>
> Are you hitting hardware limitations and perhaps swapping like
> crazy? In which case, unless you split things across several
> machines, I doubt it would help to make two smaller indexes.
>
> In sum, I really suspect that you're NOT hitting a Lucene limitation,
> but it's something else about your system....
>
> Best
> Erick
>
> On 3/28/07, Scott Oshima <soshima@gmail.com> wrote:
>>
>> So I assumed a linear decay of performance as an index got bigger.
>>
>> For some reason when going from an index size of 1.89 to 1.95 gigs
>> dramatically increased cpu across all of our servers.
>>
>> I was thinking of splitting the 1.95 index into 2 separate  
>> indexes  and
>> using a multisearcher on those parts?
>>
>> thanks.
>>
>> -scott
>>


---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org


Mime
View raw message