Mailing-List: contact java-user-help@lucene.apache.org; run by ezmlm
Precedence: bulk
Reply-To: java-user@lucene.apache.org
Received-SPF: pass (asf.osuosl.org: local policy)
DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws;
  s=s1024; d=yahoo.com;
  h=Message-ID:Date:From:Reply-To:Subject:To:In-Reply-To:MIME-Version:Content-Type;
  b=aP68cplZqUuExQ7AqWcr/cTz4pULQNflzcjnJMYjKiO5nxXZ+HNUzuW7kVxEk7tm8YjLwLlwiiTA9CgZVNTJXXtRlm1WbYqdTSswx1MM44voz60cX5UiSkZsxN0R6U3piLvk+ofRC/rCxZebLJq+pUhKWXx9a5i/Y6Gssu/tiOI=
  ;
Message-ID: <20060222214635.83398.qmail@web50310.mail.yahoo.com>
Date: Wed, 22 Feb 2006 13:46:35 -0800 (PST)
From: Otis Gospodnetic <otis_gospodnetic@yahoo.com>
Reply-To: Otis Gospodnetic <otis_gospodnetic@yahoo.com>
Subject: Re: Throughput doesn't increase when using more concurrent threads
To: java-user@lucene.apache.org
In-Reply-To: <c68e39170602221218g6f7c2cb2m225c16f668b8497b@mail.gmail.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii

Hi,

Some things that could be different:
- thread scheduling (shouldn't make too much of a difference though)

--- I would also play with disk IO schedulers, if you can.  CentOS is based on RedHat, I believe, and RedHat (ext3, really) now has about 4 different IO schedulers that, according to articles I recently read, can have an impact on disk read/write performance.  These schedules can be specified at mount time, I believe, and maybe at boot time (kernel line in Grub/LILO).

Otis


On 2/22/06, Peter Keegan <peterlkeegan@gmail.com> wrote:
> I am doing a performance comparison of Lucene on Linux vs Windows.
>
> I have 2 identically configured servers (8-CPUs (real) x 3GHz Xeon
> processors, 64GB RAM). One is running CentOS 4 Linux, the other is running
> Windows server 2003 Enterprise Edition x64. Both have 64-bit JVMs from Sun.
> The Lucene server is using MMapDirectory. I'm running the jvm with
> -Xmx16000M. Peak memory usage of the jvm on Linux is about 6GB and 7.8GB on
> windows.
>
> I'm observing query rates of 330 queries/sec on the Wintel server, but only
> 200 qps on the Linux box. At first, I suspected a network bottleneck, but
> when I 'short-circuited' Lucene, the query rates were identical.
>
> I suspect that there are some things to be tuned in Linux, but I'm not sure
> what. Any advice would be appreciated.
>
> Peter
>
>
>
> On 1/30/06, Peter Keegan <peterlkeegan@gmail.com> wrote:
> >
> > I cranked up the dial on my query tester and was able to get the rate up
> > to 325 qps. Unfortunately, the machine died shortly thereafter (memory
> > errors :-( ) Hopefully, it was just a coincidence. I haven't measured 64-bit
> > indexing speed, yet.
> >
> > Peter
> >
> > On 1/29/06, Daniel Noll <daniel@nuix.com.au> wrote:
> > >
> > > Peter Keegan wrote:
> > > > I tried the AMD64-bit JVM from Sun and with MMapDirectory and I'm now
> > > > getting 250 queries/sec and excellent cpu utilization (equal
> > > concurrency on
> > > > all cpus)!! Yonik, thanks for the pointer to the 64-bit jvm. I wasn't
> > > aware
> > > > of it.
> > > >
> > > Wow.  That's fast.
> > >
> > > Out of interest, does indexing time speed up much on 64-bit hardware?
> > > I'm particularly interested in this side of things because for our own
> > > application, any query response under half a second is good enough, but
> > > the indexing side could always be faster. :-)
> > >
> > > Daniel
> > >
> > > --
> > > Daniel Noll
> > >
> > > Nuix Australia Pty Ltd
> > > Suite 79, 89 Jones St, Ultimo NSW 2007, Australia
> > > Phone: (02) 9280 0699
> > > Fax:   (02) 9212 6902
> > >
> > > This message is intended only for the named recipient. If you are not
> > > the intended recipient you are notified that disclosing, copying,
> > > distributing or taking any action in reliance on the contents of this
> > > message or attachment is strictly prohibited.
> > >
> > >
> > > ---------------------------------------------------------------------
> > > To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> > > For additional commands, e-mail: java-user-help@lucene.apache.org
> > >
> > >
> >
>
>

---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org