lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From karl wettin <karl.wet...@gmail.com>
Subject Re: Amount of RAM needed to support a growing lucene index?
Date Sun, 12 Aug 2007 12:22:28 GMT

12 aug 2007 kl. 14.01 skrev lucene user:

> Do you know if 290k articles and 234 million words is a large  
> lucene index
> or a medium one? Do people build them this big all the time?

If the calculator in my head works you have 300k documents at 4k text  
each.

I say your corpus is borderline small.


-- 
karl

> Thanks!
>
> On 8/12/07, karl wettin <karl.wettin@gmail.com> wrote:
>>
>>
>> 12 aug 2007 kl. 09.03 skrev lucene user:
>>
>>> If I have an index with 111k articles and 90 million words indexed,
>>> how much
>>> RAM should I have to get really fast access speeds?
>>>
>>> If I have an index with 290k articles and 234 million words
>>> indexed, how
>>> much RAM should I have to get really fast access speeds?
>>
>> Define really fast.
>>
>> I say you need 1.3x as much RAM as the size of your FSDirectory to
>> ensure that the file system cache is never flushed out. But it also
>> depends on user load. Each thread consumes RAM and CPU.
>>
>> In order to really find out, setup the benchmarker to run on your
>> index, and limit the amount of memory your file system chache and JVM
>> is allowed.
>>
>>> Any other advice about sizing a server?
>>> What other info do you need to have to help size the server?
>>
>> Sizing?
>>
>>> Does it matter if the server has a 64 bit processor?
>>
>> In a 64 bit environment a reference to an instance consumes twice as
>> much RAM as in a 32 bit environment. It should not affect a file
>> centric Lucene store (Directory), Your OS and your application that
>> use Lucene might be consuming some more resources though. Again,
>> benchmark.
>>
>>> Speed of processor important?
>>
>> Yes.
>>
>>> Speed of disks?
>>
>> May or may not be intersting depending on how much RAM you have.
>>
>>
>>
>> --
>> karl
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
>> For additional commands, e-mail: java-user-help@lucene.apache.org
>>
>>


---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org


Mime
View raw message