jackrabbit-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "David Johnson" <dbjohnso...@gmail.com>
Subject Re: Threading and Query Performance
Date Tue, 13 Mar 2007 21:05:11 GMT
So if I increase the # of threads to a worst case scenario - 100 threads
running 1 query each - I would think I would see a worst case in
synchronization and scheduling overhead - i.e., 100 threads would run
significantly slower than the 25 thread run.

The 100 thread run took 37070 ms to run.  Not that far off from the 25
thread run.

Only a 10% speed increase between the 1 thread and 2 thread seems off
(running on a 4 core box) - unless, as you mention, the searching is hitting
a  synchronization issue.  I am no threading expert by any means, it just
seems off.

Again, there are no writes happening to this repository during these tests -
it is a read only repository.

-Dave

On 3/13/07, Tobias Bocanegra <tobias.bocanegra@day.com> wrote:
>
> well, from a first glance: the more threads you add, the faster the
> queries are until you reach the number of processors. if you expect
> linear improvement - that does not work, since the actual 'searching'
> is synchronized (marcel, correct me if i'm saying something wrong
> here).
> when you add more ad more threads, the synchronization and scheduling
> overhead gets bigger and you loose again overall speed.
>
> what you must ask yourself: is '~350ms' per query fast enough for my
> application? how do my queries look like? can i optimize on the query
> or data structure?
>
> regards, toby
>
> On 3/12/07, David Johnson <dbjohnson.e@gmail.com> wrote:
> > This is related to two ongoing list threads - one on synchronization and
> the
> > other on query performance.
> >
> > As I have mentioned in previous posts, I have been running a variety of
> > query tests.  I am using a suite of 100 queries and running them against
> > Jackrabbit in several different threading scenarios - i.e., I change the
> #
> > of threads used to run sub-sets of the 100 queries.  To be clear - if I
> run
> > a single thread case, it will run all 100 queries, one after the
> other.  If
> > I run 2 threads - one thread will run 50 queries, while the other thread
> > will run the other 50 queries.  In all cases, the 100 queries are the
> same,
> > the only thing that changes is the number of threads used to run them.
> > Also, in all tests, the repository is read only - nothing is making any
> > writes to the repository.
> >
> > Here are some results:
> >
> > 1 thread: 100 queries in 41139 ms
> > 2 threads: 50 queries in 37828 ms, 50 queries in 38622 ms - total time
> for
> > all threads to complete 38960 ms
> > 4 threads: 25 queries in 25895 ms, 25 queries in 28034 ms, 25 queries in
> > 32335 ms, 25 queries in 32391 ms - total time 32801 ms
> > 10 threads: 10 queries in 18733 ms, 10 queries in 19894 ms, ... , 10
> queries
> > in 33798 ms, 10 queries in 34924 ms - total time 35286 ms
> > 25 threads: 4 queries in 2413 ms, 4 queries in 11725 ms, 4 queries in
> 18294
> > ms, ... , 4 queries in 36059 ms, 4 queries in 36222 ms
> >
> > Some details on the box that I am running these tests on: it is a dual
> Xeon
> > running Linux - /proc/cpuinfo shows 4 processors, so I am assuming it is
> a
> > dual core.  I am running Jackrabbit 1.2.3 with the Bundle Persistence
> > Manager.
> >
> > I am not sure what the numbers above are really saying, although they
> don't
> > really look right :-)  We have a multi-user use case - large web site
> with
> > many ongoing reads, occasional writes.  I am using the multiple threads
> to
> > "test" multiple users.  I am hoping that the developers with more
> > understanding of the internals can explain what's going on above.
> >
> > I am wondering if I am hitting the synchronization issue that is being
> > discussed in other posts?  Thoughts?
> >
> > -Dave
> >
>
>
> --
> -----------------------------------------< tobias.bocanegra@day.com >---
> Tobias Bocanegra, Day Management AG, Barfuesserplatz 6, CH - 4001 Basel
> T +41 61 226 98 98, F +41 61 226 98 97
> -----------------------------------------------< http://www.day.com >---
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message