lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Yonik Seeley" <ysee...@gmail.com>
Subject Re: Throughput doesn't increase when using more concurrent threads
Date Wed, 22 Feb 2006 20:18:26 GMT
Hmmm, not sure what that could be.
You could try using the default FSDir instead of MMapDir to see if the
differences are there.

Some things that could be different:
- thread scheduling (shouldn't make too much of a difference though)
- synchronization workings
- page replacement policy... how to figure out what pages to swap in
and which to swap out, esp of the memory mapped files.

You could also try a profiler on both platforms to try and see where
the difference is.

-Yonik

On 2/22/06, Peter Keegan <peterlkeegan@gmail.com> wrote:
> I am doing a performance comparison of Lucene on Linux vs Windows.
>
> I have 2 identically configured servers (8-CPUs (real) x 3GHz Xeon
> processors, 64GB RAM). One is running CentOS 4 Linux, the other is running
> Windows server 2003 Enterprise Edition x64. Both have 64-bit JVMs from Sun.
> The Lucene server is using MMapDirectory. I'm running the jvm with
> -Xmx16000M. Peak memory usage of the jvm on Linux is about 6GB and 7.8GB on
> windows.
>
> I'm observing query rates of 330 queries/sec on the Wintel server, but only
> 200 qps on the Linux box. At first, I suspected a network bottleneck, but
> when I 'short-circuited' Lucene, the query rates were identical.
>
> I suspect that there are some things to be tuned in Linux, but I'm not sure
> what. Any advice would be appreciated.
>
> Peter
>
>
>
> On 1/30/06, Peter Keegan <peterlkeegan@gmail.com> wrote:
> >
> > I cranked up the dial on my query tester and was able to get the rate up
> > to 325 qps. Unfortunately, the machine died shortly thereafter (memory
> > errors :-( ) Hopefully, it was just a coincidence. I haven't measured 64-bit
> > indexing speed, yet.
> >
> > Peter
> >
> > On 1/29/06, Daniel Noll <daniel@nuix.com.au> wrote:
> > >
> > > Peter Keegan wrote:
> > > > I tried the AMD64-bit JVM from Sun and with MMapDirectory and I'm now
> > > > getting 250 queries/sec and excellent cpu utilization (equal
> > > concurrency on
> > > > all cpus)!! Yonik, thanks for the pointer to the 64-bit jvm. I wasn't
> > > aware
> > > > of it.
> > > >
> > > Wow.  That's fast.
> > >
> > > Out of interest, does indexing time speed up much on 64-bit hardware?
> > > I'm particularly interested in this side of things because for our own
> > > application, any query response under half a second is good enough, but
> > > the indexing side could always be faster. :-)
> > >
> > > Daniel
> > >
> > > --
> > > Daniel Noll
> > >
> > > Nuix Australia Pty Ltd
> > > Suite 79, 89 Jones St, Ultimo NSW 2007, Australia
> > > Phone: (02) 9280 0699
> > > Fax:   (02) 9212 6902
> > >
> > > This message is intended only for the named recipient. If you are not
> > > the intended recipient you are notified that disclosing, copying,
> > > distributing or taking any action in reliance on the contents of this
> > > message or attachment is strictly prohibited.
> > >
> > >
> > > ---------------------------------------------------------------------
> > > To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> > > For additional commands, e-mail: java-user-help@lucene.apache.org
> > >
> > >
> >
>
>

---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org


Mime
View raw message