lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Robert Engels" <reng...@ix.netcom.com>
Subject RE: FieldsReader synchronized access vs. ThreadLocal ?
Date Fri, 19 May 2006 16:04:47 GMT
fyi, when the OS disk cache is not primed, I get better performance from
more threads (since when one thread is blocked on IO, the other may find its
data available, and continue the query), but as the cache becomes "ready"
the more threads I use the worse the performance (due to the overhead of
context switching). I am going to be able to test very soon on some
multi-processor Sun opteron machines, so I'll update when the information is
available.

It seems the best solution might be to use an adaptive pool, where based on
query avg. query response time the number of processing threads is adjusted.
Not sure if it would work reliably (since it would have to ignore complex
queries that might bias the assessment).

Our "search server" runs in a separate process and uses its own pool of
worker threads, so it would be quite easy for us to test something like
this.

I guess this is another reason that a 'core' set of performance tests, with
index corpuses, would be ideal in order to evaluate how different
modifications work based on hardware/OS.

-----Original Message-----
From: Robert Engels [mailto:rengels@ix.netcom.com]
Sent: Friday, May 19, 2006 9:18 AM
To: java-dev@lucene.apache.org
Subject: RE: FieldsReader synchronized access vs. ThreadLocal ?


If you can get much higher throughput from a single threaded client, then
you should queue the requests in the server and process them from a single
thread (or a small pool of threads).

If you can get much higher throughput from a single threaded client seems to
also imply that Lucene (or at least your packaging) is NOT very concurrent
(since more threads actually reduce the efficiency).

-----Original Message-----
From: Peter Keegan [mailto:peterlkeegan@gmail.com]
Sent: Friday, May 19, 2006 9:03 AM
To: java-dev@lucene.apache.org; rengels@ix.netcom.com
Subject: Re: FieldsReader synchronized access vs. ThreadLocal ?


The queries are mostly boolean (all AND'd terms), no. of terms varies
anywhere from a few to 25 or more, with 1 or 2 sort fields.

My tests are designed to measure total query throughput, not just raw search
speed. The client test program blasts queries from >50 threads over a socket
and runs on a separate server from Lucene. I can get much higher rates by
just blasting from a single thread in the client, but this doesn't simulate
the real use model.

Peter

On 5/19/06, Robert Engels <rengels@ix.netcom.com> wrote:
>
> As an aside,
>
> On my VERY crappy 1.2 ghz single CPU P4, using a index of 300k documents,
> I
> can perform 50 searches per second (returning 20 document matches each).
> This includes the time to serialize and send the results to the client
> (although the client is on the same machine, but it also competes for cpu
> with the search server).
>
> Based on some informal viewing of the CPU usage, the client consume 50-70%
> of the cpu, so I would assume that moving the client off the server should
> double the queries per second (although there would be additional delay
> due
> to network transmission). So even a single CPU P4 could easily do 100
> queries per second.
>
> Even though we are comparing apples and oranges, unless you are performing
> some really expensive queries, I would expect your configuration to be
> MUCH
> faster.
>
> -----Original Message-----
> From: Peter Keegan [mailto:peterlkeegan@gmail.com]
> Sent: Thursday, May 18, 2006 1:32 PM
> To: java-dev@lucene.apache.org; rengels@ix.netcom.com
> Subject: Re: FieldsReader synchronized access vs. ThreadLocal ?
>
>
> I'm returning 20 results (about .5Kb each). In fact, I had to reduce that
> from 50 because the network was becoming the bottleneck.
>
> On the 16-cpu server, I ran tests using 8, 16 and 32 query threads, but
> there was no improvement with more threads. I still believe the hardware
> was
> to blame.
>
> Peter
>
> On 5/18/06, Robert Engels <rengels@ix.netcom.com> wrote:
> >
> > As someone else pointed out, the proposed mods will only affect queries
> > the
> > return a lot of Documents. If your test is only set up to return a few
> > documents (or possible none at all), then you will see no difference.
> >
> > The fact that some of the CPUs were far less than 100%, and others were
> at
> > 100% may be a good sign. How any query threads were you testing with?
> >
> > -----Original Message-----
> > From: Peter Keegan [mailto:peterlkeegan@gmail.com]
> > Sent: Thursday, May 18, 2006 1:01 PM
> > To: java-dev@lucene.apache.org; rengels@ix.netcom.com
> > Subject: Re: FieldsReader synchronized access vs. ThreadLocal ?
> >
> >
> > Robert,
> >
> > Sorry I missed your questions.
> >
> > The test results seem hard to believe. Doubling the CPUs only increased
> > > through put by 20%??? Seems rather low for primarily a "read only"
> test.
> >
> >
> > I think this refers to the test I did on a 16 cpu (32 hyperthreaded)
> > server.
> > This system was actually two 8 cpu systems cabled together on their
> > backplanes. I suspect that some tradeoffs were made in its design that
> > allowed for this flexibility which resulted in the minimal improvement
> in
> > the tests.
> >
> > Peter did not seem to answer many of the follow-up questions (at least I
> > > could not find the answers) regarding whether or not the CPU usage was
> > > 100%.
> >
> >
> > On the 16-cpu system I noticed that load was not distributed very evenly
> -
> > some were near 100%, others were less than 10%. On the AMD Opteron
> > servers,
> > the distribution was quite even and between 75-100%.
> >
> > I look forward to your thoughts, and others - hopefully someone can run
> > the
> > > test on a multiple CPU machine.
> > >
> > >
> > I built Lucene with your mod's and ran my test on the 8 cpu AMD Linux
> > server, but noticed no difference in query throughput. It would seem
> that
> > ThreadLocal could improve performance, but I think my bottlenecks are
> > elsewhere, like IndexInput.readVInt and inserting results in priority
> > queues.
> >
> > Peter
> >
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
> > For additional commands, e-mail: java-dev-help@lucene.apache.org
> >
> >
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-dev-help@lucene.apache.org
>
>


---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org


Mime
View raw message