lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Kai_testing Middleton <kai_test...@yahoo.com>
Subject Re: Amount of RAM needed to support a growing lucene index?
Date Mon, 13 Aug 2007 22:22:26 GMT
I don't think that's all that large, though I have only been working with Lucene for a short
while.  I have two corpuses with 445834 documents (3.43M terms) and 132217 documents (1.6M
terms).  I don't have trouble querying either of these with Luke.

----- Original Message ----
From: lucene user <luz290@gmail.com>
To: java-user@lucene.apache.org
Sent: Sunday, August 12, 2007 5:01:28 AM
Subject: Re: Amount of RAM needed to support a growing lucene index?

Thanks, Karl.

Do you know if 290k articles and 234 million words is a large lucene index
or a medium one? Do people build them this big all the time?

Thanks!

On 8/12/07, karl wettin <karl.wettin@gmail.com> wrote:
>
>
> 12 aug 2007 kl. 09.03 skrev lucene user:
>
> > If I have an index with 111k articles and 90 million words indexed,
> > how much
> > RAM should I have to get really fast access speeds?
> >
> > If I have an index with 290k articles and 234 million words
> > indexed, how
> > much RAM should I have to get really fast access speeds?
>
> Define really fast.
>
> I say you need 1.3x as much RAM as the size of your FSDirectory to
> ensure that the file system cache is never flushed out. But it also
> depends on user load. Each thread consumes RAM and CPU.
>
> In order to really find out, setup the benchmarker to run on your
> index, and limit the amount of memory your file system chache and JVM
> is allowed.
>
> > Any other advice about sizing a server?
> > What other info do you need to have to help size the server?
>
> Sizing?
>
> > Does it matter if the server has a 64 bit processor?
>
> In a 64 bit environment a reference to an instance consumes twice as
> much RAM as in a 32 bit environment. It should not affect a file
> centric Lucene store (Directory), Your OS and your application that
> use Lucene might be consuming some more resources though. Again,
> benchmark.
>
> > Speed of processor important?
>
> Yes.
>
> > Speed of disks?
>
> May or may not be intersting depending on how much RAM you have.
>
>
>
> --
> karl
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org
>
>







       
____________________________________________________________________________________
Be a better Heartthrob. Get better relationship answers from someone who knows. Yahoo! Answers
- Check it out. 
http://answers.yahoo.com/dir/?link=list&sid=396545433
Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message