lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From eks dev <eks...@yahoo.co.uk>
Subject Re: Amount of RAM needed to support a growing lucene index?
Date Sun, 12 Aug 2007 12:39:38 GMT
300k documents is something I would consider very small. Anything under 10Mio documents IMHO
is small for Lucene (meaning, commodity hardware, 1G RAM should give you well under second
response times).
The number of words is not all that important, much more important would be the number of
unique words.

----- Original Message ----
From: lucene user <luz290@gmail.com>
To: java-user@lucene.apache.org
Sent: Sunday, 12 August, 2007 2:01:28 PM
Subject: Re: Amount of RAM needed to support a growing lucene index?

Thanks, Karl.

Do you know if 290k articles and 234 million words is a large lucene index
or a medium one? Do people build them this big all the time?

Thanks!

On 8/12/07, karl wettin <karl.wettin@gmail.com> wrote:
>
>
> 12 aug 2007 kl. 09.03 skrev lucene user:
>
> > If I have an index with 111k articles and 90 million words indexed,
> > how much
> > RAM should I have to get really fast access speeds?
> >
> > If I have an index with 290k articles and 234 million words
> > indexed, how
> > much RAM should I have to get really fast access speeds?
>
> Define really fast.
>
> I say you need 1.3x as much RAM as the size of your FSDirectory to
> ensure that the file system cache is never flushed out. But it also
> depends on user load. Each thread consumes RAM and CPU.
>
> In order to really find out, setup the benchmarker to run on your
> index, and limit the amount of memory your file system chache and JVM
> is allowed.
>
> > Any other advice about sizing a server?
> > What other info do you need to have to help size the server?
>
> Sizing?
>
> > Does it matter if the server has a 64 bit processor?
>
> In a 64 bit environment a reference to an instance consumes twice as
> much RAM as in a 32 bit environment. It should not affect a file
> centric Lucene store (Directory), Your OS and your application that
> use Lucene might be consuming some more resources though. Again,
> benchmark.
>
> > Speed of processor important?
>
> Yes.
>
> > Speed of disks?
>
> May or may not be intersting depending on how much RAM you have.
>
>
>
> --
> karl
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org
>
>





      ___________________________________________________________
Yahoo! Answers - Got a question? Someone out there knows the answer. Try it
now.
http://uk.answers.yahoo.com/ 

---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org


Mime
View raw message