db-derby-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Knut Anders Hatlen (JIRA)" <j...@apache.org>
Subject [jira] Updated: (DERBY-2911) Implement a buffer manager using java.util.concurrent classes
Date Fri, 28 Sep 2007 15:54:50 GMT

     [ https://issues.apache.org/jira/browse/DERBY-2911?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel

Knut Anders Hatlen updated DERBY-2911:

    Attachment: perftest6.pdf

I mentioned that I ran some performance tests with the new buffer manager after patch #6.
Here's a pdf (perftest6.pdf) with the details for those interested in graphs and numbers.
I have used the test client attached to this issue and the one attached to DERBY-1961, so
there are numbers for single-record select operations, update operations and two-way joins.
To test the replacement algorithm, I varied the page cache size from 40 up to 40000.

The tests were run both on an AMD Opteron with two 2.4 GHz CPUs, and on a Sun Fire T2000,
1.0 GHz with 8 cores/32 hardware threads.

The test client attached to this issue was used to show the scalability of the buffer manager.
The DERBY-1961 client was used primarily to test the replacement algorithm, since that client
scales poorly because of latch contention on the root node of the B-tree (all threads work
on the same table).

For the details, see the pdf, but I'm pasting the conclusion here:

For most of the loads used in these tests, the new buffer manager performs as good as or better
than the old buffer manager. There was one exception: single-record update on Opteron with
page cache size 40000. That test was not conclusive, though, as the single run which had highest
throughput in that configuration was the new buffer manager with 60 concurrent clients. Overall,
the new buffer manager seems to increase the performance.

Note that the new buffer manager does not yet implement a background cleaner, which is supposed
to increase the likelihood of finding a clean page to evict and thereby making the page replacement
algorithm more effective. It is not clear to me whether the existence of a background cleaner
affects the performance of these tests.

> Implement a buffer manager using java.util.concurrent classes
> -------------------------------------------------------------
>                 Key: DERBY-2911
>                 URL: https://issues.apache.org/jira/browse/DERBY-2911
>             Project: Derby
>          Issue Type: Improvement
>          Components: Performance, Services
>    Affects Versions:
>            Reporter: Knut Anders Hatlen
>            Assignee: Knut Anders Hatlen
>            Priority: Minor
>         Attachments: d2911-1.diff, d2911-1.stat, d2911-2.diff, d2911-3.diff, d2911-4.diff,
d2911-5.diff, d2911-6.diff, d2911-6.stat, d2911-entry-javadoc.diff, d2911-unused.diff, d2911-unused.stat,
d2911perf.java, perftest6.pdf
> There are indications that the buffer manager is a bottleneck for some types of multi-user
load. For instance, Anders Morken wrote this in a comment on DERBY-1704: "With a separate
table and index for each thread (to remove latch contention and lock waits from the equation)
we (...) found that org.apache.derby.impl.services.cache.Clock.find()/release() caused about
5 times more contention than the synchronization in LockSet.lockObject() and LockSet.unlock().
That might be an indicator of where to apply the next push".
> It would be interesting to see the scalability and performance of a buffer manager which
exploits the concurrency utilities added in Java SE 5.

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message