lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From eks dev <eks...@yahoo.co.uk>
Subject Re: Post mortem kudos for (LUCENE-843) :)
Date Fri, 13 Jul 2007 14:48:19 GMT
no, we have only 2G on this machine and 32bit jvm can utilize most of it (1..6G) leaving something
to the OS, and is generally faster due to shorter pointers and more availyble registers on
32bit. Lucene is not memory hungry for *our setup* and works perfectly fine with 32bit jvm...

actually, our experience with lucene based apps showed it is often  better to give more memory
to OS and less to the jvm so the OS file cache can show its magic. Indexing works more than
happily with -Xmx512m



----- Original Message ----
From: Peter Keegan <peterlkeegan@gmail.com>
To: java-dev@lucene.apache.org
Sent: Friday, 13 July, 2007 4:39:45 PM
Subject: Re: Post mortem kudos for (LUCENE-843) :)

I see you have a 64-bit OS but 32-bit JVM. Have you tried a 64-bit JVM (with
a bigger RAM buffer)?

Peter


On 7/13/07, eks dev <eksdev@yahoo.co.uk> wrote:
>
> >Interesting.  This matches the experience Doron had where adding more
> >RAM actually slowed things down a bit (posted to
> >LUCENE-843).
>
> I know this intrigues you, so our fresh experience:
>
> The bigger the RAM Buffer the faster indexing, this holds until you hit
> some limit that starts irritating gc(). But this limit  is somehow "natural"
> and is given by the environment (available RAM, competing OS File cache and
> who knows what else)... basically , we concluded  after testing the more
> memory, more speed is to expect (this is kind of ideal scaling, one  proof
> more of the algorithmic strength).
>
> This test showed 110k Docs/second at 32Mb (what we found to be optimal for
> our needs, as it slowly speeds-up after that to 123k Docs/sec at 256Mb)
>
> I suspect this phenomena on our last test and what Doron mentioned was due
> to the wrong maxBufferedDocs. Have no other explanation
>
> Basically, we achieved almost 20 X speed-up by just having LUCEN-843 and
> your valuable comments on how to utilize this nice machine called Lucene.
>
>
>
>
>
>
>
>
>       ___________________________________________________________
> Yahoo! Mail is the world's favourite email. Don't settle for less, sign up
> for
> your free account today
> http://uk.rd.yahoo.com/evt=44106/*http://uk.docs.yahoo.com/mail/winter07.html
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-dev-help@lucene.apache.org
>
>





      ___________________________________________________________ 
Yahoo! Mail is the world's favourite email. Don't settle for less, sign up for
your free account today http://uk.rd.yahoo.com/evt=44106/*http://uk.docs.yahoo.com/mail/winter07.html



---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org


Mime
View raw message