lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Peter Keegan" <peterlkee...@gmail.com>
Subject Re: FieldsReader synchronized access vs. ThreadLocal ?
Date Thu, 18 May 2006 18:32:20 GMT
I'm returning 20 results (about .5Kb each). In fact, I had to reduce that
from 50 because the network was becoming the bottleneck.

On the 16-cpu server, I ran tests using 8, 16 and 32 query threads, but
there was no improvement with more threads. I still believe the hardware was
to blame.

Peter

On 5/18/06, Robert Engels <rengels@ix.netcom.com> wrote:
>
> As someone else pointed out, the proposed mods will only affect queries
> the
> return a lot of Documents. If your test is only set up to return a few
> documents (or possible none at all), then you will see no difference.
>
> The fact that some of the CPUs were far less than 100%, and others were at
> 100% may be a good sign. How any query threads were you testing with?
>
> -----Original Message-----
> From: Peter Keegan [mailto:peterlkeegan@gmail.com]
> Sent: Thursday, May 18, 2006 1:01 PM
> To: java-dev@lucene.apache.org; rengels@ix.netcom.com
> Subject: Re: FieldsReader synchronized access vs. ThreadLocal ?
>
>
> Robert,
>
> Sorry I missed your questions.
>
> The test results seem hard to believe. Doubling the CPUs only increased
> > through put by 20%??? Seems rather low for primarily a "read only" test.
>
>
> I think this refers to the test I did on a 16 cpu (32 hyperthreaded)
> server.
> This system was actually two 8 cpu systems cabled together on their
> backplanes. I suspect that some tradeoffs were made in its design that
> allowed for this flexibility which resulted in the minimal improvement in
> the tests.
>
> Peter did not seem to answer many of the follow-up questions (at least I
> > could not find the answers) regarding whether or not the CPU usage was
> > 100%.
>
>
> On the 16-cpu system I noticed that load was not distributed very evenly -
> some were near 100%, others were less than 10%. On the AMD Opteron
> servers,
> the distribution was quite even and between 75-100%.
>
> I look forward to your thoughts, and others - hopefully someone can run
> the
> > test on a multiple CPU machine.
> >
> >
> I built Lucene with your mod's and ran my test on the 8 cpu AMD Linux
> server, but noticed no difference in query throughput. It would seem that
> ThreadLocal could improve performance, but I think my bottlenecks are
> elsewhere, like IndexInput.readVInt and inserting results in priority
> queues.
>
> Peter
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-dev-help@lucene.apache.org
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message