Mailing-List: contact java-user-help@lucene.apache.org; run by ezmlm
Precedence: bulk
Reply-To: java-user@lucene.apache.org
Received-SPF: pass (asf.osuosl.org: domain of peterlkeegan@gmail.com
 designates 64.233.184.199 as permitted sender)
DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws;
        s=beta; d=gmail.com;
        h=received:message-id:date:from:to:subject:in-reply-to:mime-version:content-type:references;
        b=NXQWbbN0LhWgvnodxPUusK1t8J/3EUkDTW2R5sqwJT3YG6CEZ+4eSLPI5CQE88PcLgvrkvCiiZCkZ9p0sk0nBZL1z60x/aXcquWO+92B1mU3MCvSx0NyRjkCJbM9aCJz1jtzVmXu/Qw2yNKu9g+drYM2orbS6vD0/Fr+py5oUWc=
Message-ID: <e994873a0602231314t5c7707bdjf79665d3d4ad1cf0@mail.gmail.com>
Date: Thu, 23 Feb 2006 16:14:24 -0500
From: "Peter Keegan" <peterlkeegan@gmail.com>
To: java-user@lucene.apache.org
Subject: Re: Throughput doesn't increase when using more concurrent threads
In-Reply-To: <88c6a6720602231148v520b3be6uab8cfd7fb48899a9@mail.gmail.com>
MIME-Version: 1.0
Content-Type: multipart/alternative;
	boundary="----=_Part_17007_16436157.1140729264160"
References: <GJEMKBFALLAJKLGINBHHKEHAKOAA.jbooth@net-temps.com>
	 <43D92935.1000809@apache.org> <43D92A28.6050501@apache.org>
	 <e994873a0601261320r753d864cwcb37b72e54539d09@mail.gmail.com>
	 <43DD6821.8020302@nuix.com.au>
	 <e994873a0601300615p5f46dab6qfb4b4504bfb2bd7f@mail.gmail.com>
	 <e994873a0602221152q1bad1e52gdeb3477611d31916@mail.gmail.com>
	 <c68e39170602221218g6f7c2cb2m225c16f668b8497b@mail.gmail.com>
	 <e994873a0602231100v48a73ec4tcca903fc65548fa6@mail.gmail.com>
	 <88c6a6720602231148v520b3be6uab8cfd7fb48899a9@mail.gmail.com>

------=_Part_17007_16436157.1140729264160
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable
Content-Disposition: inline

Chris,

I tried JRockit a while back on 8-cpu/windows and it was slower than Sun's.
Since I seem to be cpu-bound right now, I'll be trying a 16-cpu system next
(32 with hyperthreading), on LinTel. I may give JRockit another go around
then.

Thanks,
Peter

On 2/23/06, Chris Lamprecht <clamprecht@gmail.com> wrote:
>
> Peter,
> Have you given JRockit JVM a try?  I've seen it help throughput
> compared to Sun's JVM on a dual xeon/linux machine, especially with
> concurrency (up to 6 concurrent searches happening).  I'm curious to
> see if it makes a difference for you.
>
> -chris
>
> On 2/23/06, Peter Keegan <peterlkeegan@gmail.com> wrote:
> > We discovered that the kernel was only using 8 CPUs. After recompiling
> for
> > 16 (8+hyperthreads), it looks like the query rate will settle in around
> > 280-300 qps. Much better, although still quite a bit slower than the
> > opteron.
> >
> > Peter
> >
> >
> >
> >
> > On 2/22/06, Yonik Seeley <yseeley@gmail.com> wrote:
> > >
> > > Hmmm, not sure what that could be.
> > > You could try using the default FSDir instead of MMapDir to see if th=
e
> > > differences are there.
> > >
> > > Some things that could be different:
> > > - thread scheduling (shouldn't make too much of a difference though)
> > > - synchronization workings
> > > - page replacement policy... how to figure out what pages to swap in
> > > and which to swap out, esp of the memory mapped files.
> > >
> > > You could also try a profiler on both platforms to try and see where
> > > the difference is.
> > >
> > > -Yonik
> > >
> > > On 2/22/06, Peter Keegan <peterlkeegan@gmail.com> wrote:
> > > > I am doing a performance comparison of Lucene on Linux vs Windows.
> > > >
> > > > I have 2 identically configured servers (8-CPUs (real) x 3GHz Xeon
> > > > processors, 64GB RAM). One is running CentOS 4 Linux, the other is
> > > running
> > > > Windows server 2003 Enterprise Edition x64. Both have 64-bit JVMs
> from
> > > Sun.
> > > > The Lucene server is using MMapDirectory. I'm running the jvm with
> > > > -Xmx16000M. Peak memory usage of the jvm on Linux is about 6GB and
> 7.8GBon
> > > > windows.
> > > >
> > > > I'm observing query rates of 330 queries/sec on the Wintel server,
> but
> > > only
> > > > 200 qps on the Linux box. At first, I suspected a network
> bottleneck,
> > > but
> > > > when I 'short-circuited' Lucene, the query rates were identical.
> > > >
> > > > I suspect that there are some things to be tuned in Linux, but I'm
> not
> > > sure
> > > > what. Any advice would be appreciated.
> > > >
> > > > Peter
> > > >
> > > >
> > > >
> > > > On 1/30/06, Peter Keegan <peterlkeegan@gmail.com> wrote:
> > > > >
> > > > > I cranked up the dial on my query tester and was able to get the
> rate
> > > up
> > > > > to 325 qps. Unfortunately, the machine died shortly thereafter
> (memory
> > > > > errors :-( ) Hopefully, it was just a coincidence. I haven't
> measured
> > > 64-bit
> > > > > indexing speed, yet.
> > > > >
> > > > > Peter
> > > > >
> > > > > On 1/29/06, Daniel Noll <daniel@nuix.com.au> wrote:
> > > > > >
> > > > > > Peter Keegan wrote:
> > > > > > > I tried the AMD64-bit JVM from Sun and with MMapDirectory and
> I'm
> > > now
> > > > > > > getting 250 queries/sec and excellent cpu utilization (equal
> > > > > > concurrency on
> > > > > > > all cpus)!! Yonik, thanks for the pointer to the 64-bit jvm. =
I
> > > wasn't
> > > > > > aware
> > > > > > > of it.
> > > > > > >
> > > > > > Wow.  That's fast.
> > > > > >
> > > > > > Out of interest, does indexing time speed up much on 64-bit
> > > hardware?
> > > > > > I'm particularly interested in this side of things because for
> our
> > > own
> > > > > > application, any query response under half a second is good
> enough,
> > > but
> > > > > > the indexing side could always be faster. :-)
> > > > > >
> > > > > > Daniel
> > > > > >
> > > > > > --
> > > > > > Daniel Noll
> > > > > >
> > > > > > Nuix Australia Pty Ltd
> > > > > > Suite 79, 89 Jones St, Ultimo NSW 2007, Australia
> > > > > > Phone: (02) 9280 0699
> > > > > > Fax:   (02) 9212 6902
> > > > > >
> > > > > > This message is intended only for the named recipient. If you
> are
> > > not
> > > > > > the intended recipient you are notified that disclosing,
> copying,
> > > > > > distributing or taking any action in reliance on the contents o=
f
> > > this
> > > > > > message or attachment is strictly prohibited.
> > > > > >
> > > > > >
> > > > > >
> > > ---------------------------------------------------------------------
> > > > > > To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> > > > > > For additional commands, e-mail:
> java-user-help@lucene.apache.org
> > > > > >
> > > > > >
> > > > >
> > > >
> > > >
> > >
> > > ---------------------------------------------------------------------
> > > To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> > > For additional commands, e-mail: java-user-help@lucene.apache.org
> > >
> > >
> >
> >
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org
>
>

------=_Part_17007_16436157.1140729264160--