lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Peter Keegan" <peterlkee...@gmail.com>
Subject Re: Post mortem kudos for (LUCENE-843) :)
Date Fri, 13 Jul 2007 14:39:45 GMT
I see you have a 64-bit OS but 32-bit JVM. Have you tried a 64-bit JVM (with
a bigger RAM buffer)?

Peter


On 7/13/07, eks dev <eksdev@yahoo.co.uk> wrote:
>
> >Interesting.  This matches the experience Doron had where adding more
> >RAM actually slowed things down a bit (posted to
> >LUCENE-843).
>
> I know this intrigues you, so our fresh experience:
>
> The bigger the RAM Buffer the faster indexing, this holds until you hit
> some limit that starts irritating gc(). But this limit  is somehow "natural"
> and is given by the environment (available RAM, competing OS File cache and
> who knows what else)... basically , we concluded  after testing the more
> memory, more speed is to expect (this is kind of ideal scaling, one  proof
> more of the algorithmic strength).
>
> This test showed 110k Docs/second at 32Mb (what we found to be optimal for
> our needs, as it slowly speeds-up after that to 123k Docs/sec at 256Mb)
>
> I suspect this phenomena on our last test and what Doron mentioned was due
> to the wrong maxBufferedDocs. Have no other explanation
>
> Basically, we achieved almost 20 X speed-up by just having LUCEN-843 and
> your valuable comments on how to utilize this nice machine called Lucene.
>
>
>
>
>
>
>
>
>       ___________________________________________________________
> Yahoo! Mail is the world's favourite email. Don't settle for less, sign up
> for
> your free account today
> http://uk.rd.yahoo.com/evt=44106/*http://uk.docs.yahoo.com/mail/winter07.html
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-dev-help@lucene.apache.org
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message