db-derby-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jørgen Løland (JIRA) <j...@apache.org>
Subject [jira] Commented: (DERBY-2911) Implement a buffer manager using java.util.concurrent classes
Date Fri, 09 Nov 2007 09:04:50 GMT

    [ https://issues.apache.org/jira/browse/DERBY-2911?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12541262

Jørgen Løland commented on DERBY-2911:

Hi Knut,

Thanks for providing these nice graphs; they make it much easier to understand the effects
of the new buffer manager. The results are dramatic in many of the illustrated cases!

Intuitively, I would say that your plan of using a more aggressive background cleaner sounds
good since user threads are not using the pages they have cleaned (although I don't understand
why not using that page is better). The graphs also clearly indicate that this approach is
better than letting the user thread do the job.

One question: what happens if the clock rotates too fast and reaches a synchronized (due to
I/O) page? Will it have to wait for the synchronization to be lifted, or can the clock jump
to the next page somehow? Since the background cleaner is used to write most of the pages,
it (the background cleaner) could potentially be far behind the clock if I understand this
correctly. If the clock is unnecessarily blocked by I/O it's game over ;)

On a side note - why are performance tests in Derby performed with back-to-back transactions
rather than using a more realistic (i.e., Poisson distributed) load? Poisson is normally used
to simulate the transaction arrival rate from many independent clients [1]. In your case,
I think the background cleaner could perform even better with this (more realistic) workload
since it would be scheduled in the short periods with slightly fewer active transactions.

[1] http://en.wikipedia.org/wiki/Poisson_process

> Implement a buffer manager using java.util.concurrent classes
> -------------------------------------------------------------
>                 Key: DERBY-2911
>                 URL: https://issues.apache.org/jira/browse/DERBY-2911
>             Project: Derby
>          Issue Type: Improvement
>          Components: Performance, Services
>    Affects Versions:
>            Reporter: Knut Anders Hatlen
>            Assignee: Knut Anders Hatlen
>            Priority: Minor
>         Attachments: cleaner.diff, cleaner.tar, d2911-1.diff, d2911-1.stat, d2911-2.diff,
d2911-3.diff, d2911-4.diff, d2911-5.diff, d2911-6.diff, d2911-6.stat, d2911-7.diff, d2911-7a.diff,
d2911-entry-javadoc.diff, d2911-unused.diff, d2911-unused.stat, d2911perf.java, perftest6.pdf
> There are indications that the buffer manager is a bottleneck for some types of multi-user
load. For instance, Anders Morken wrote this in a comment on DERBY-1704: "With a separate
table and index for each thread (to remove latch contention and lock waits from the equation)
we (...) found that org.apache.derby.impl.services.cache.Clock.find()/release() caused about
5 times more contention than the synchronization in LockSet.lockObject() and LockSet.unlock().
That might be an indicator of where to apply the next push".
> It would be interesting to see the scalability and performance of a buffer manager which
exploits the concurrency utilities added in Java SE 5.

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message