db-derby-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Knut Anders Hatlen (JIRA)" <j...@apache.org>
Subject [jira] Commented: (DERBY-2911) Implement a buffer manager using java.util.concurrent classes
Date Fri, 09 Nov 2007 11:10:51 GMT

    [ https://issues.apache.org/jira/browse/DERBY-2911?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12541291
] 

Knut Anders Hatlen commented on DERBY-2911:
-------------------------------------------

Hi Jørgen,

Thanks for taking the time to comment on this issue.

> One question: what happens if the clock rotates too fast and reaches a synchronized (due
to I/O) page? Will it have to wait for the synchronization to be lifted, or can the clock
jump to the next page somehow?

It will have to wait, and that may quite likely have happened in the runs with a small page
cache. It would indeed be interesting to see how the performance is affected if we allowed
the clock to jump to the next page in that case. By the way, that's what the old buffer manager
does, but for reasons mentioned in a previous comment (simplicity of implementation and less
likelihood of blocking because of finer sync granularity) ConcurrentCache doesn't. Based on
the results for update load on large db, I think it's a good idea to run some tests to see
if that was a good design decision. It shouldn't be too hard to make that change.

> why are performance tests in Derby performed with back-to-back transactions rather than
using a more realistic (i.e., Poisson distributed) load?

No particular reason, I think. And I think it's a very good idea to try with a Poisson distribution
or similar, especially to test the background cleaner. If you know any open source test clients
that could be used, I'd be more than happy to try them out. Otherwise, I may see if I can
write one myself.

> Implement a buffer manager using java.util.concurrent classes
> -------------------------------------------------------------
>
>                 Key: DERBY-2911
>                 URL: https://issues.apache.org/jira/browse/DERBY-2911
>             Project: Derby
>          Issue Type: Improvement
>          Components: Performance, Services
>    Affects Versions: 10.4.0.0
>            Reporter: Knut Anders Hatlen
>            Assignee: Knut Anders Hatlen
>            Priority: Minor
>         Attachments: cleaner.diff, cleaner.tar, d2911-1.diff, d2911-1.stat, d2911-2.diff,
d2911-3.diff, d2911-4.diff, d2911-5.diff, d2911-6.diff, d2911-6.stat, d2911-7.diff, d2911-7a.diff,
d2911-entry-javadoc.diff, d2911-unused.diff, d2911-unused.stat, d2911perf.java, perftest6.pdf
>
>
> There are indications that the buffer manager is a bottleneck for some types of multi-user
load. For instance, Anders Morken wrote this in a comment on DERBY-1704: "With a separate
table and index for each thread (to remove latch contention and lock waits from the equation)
we (...) found that org.apache.derby.impl.services.cache.Clock.find()/release() caused about
5 times more contention than the synchronization in LockSet.lockObject() and LockSet.unlock().
That might be an indicator of where to apply the next push".
> It would be interesting to see the scalability and performance of a buffer manager which
exploits the concurrency utilities added in Java SE 5.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message