db-derby-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Knut Anders Hatlen (JIRA)" <j...@apache.org>
Subject [jira] Commented: (DERBY-2911) Implement a buffer manager using java.util.concurrent classes
Date Mon, 12 Nov 2007 10:55:51 GMT

    [ https://issues.apache.org/jira/browse/DERBY-2911?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12541737

Knut Anders Hatlen commented on DERBY-2911:

> Intuitively, I would say that your plan of using a more aggressive
> background cleaner sounds good since user threads are not using the
> pages they have cleaned (although I don't understand why not using
> that page is better).

I also have a hard time understanding why not using the page after
writing it out is better, but I think it may be a result of a
double-buffering effect together with the file system cache. When a
page is not necessarily evicted after it has been cleaned (the
behaviour of Clock in trunk), a user thread may clean multiple pages
before it is able to evict a page. That could lead to more clustered
writes and, since the writes go via the fs cache and not directly to
the disk, it might increase the probability of the fs cache being able
to clean more pages in one single disk operation.

To test this, I reran the tests with direct I/O enabled on the data
disk (which disables the fs cache). I still saw that all approaches
which frequently ended up not evicting the pages they had cleaned had
about the same performance as Clock in trunk. What had changed was
that evicting the pages that had been cleaned, seemed to be slightly
better than not evicting them.

Since not evicting the pages is what most closely resembles the
behaviour of the current buffer manager, and it sounds more likely
that users have the fs cache enabled than disabled, I think I'll go
for that solution. Also, the more aggressive background cleaner seems
to perform better then the old one, so I'll try to come up with a
patch which combines those two approaches.

> Implement a buffer manager using java.util.concurrent classes
> -------------------------------------------------------------
>                 Key: DERBY-2911
>                 URL: https://issues.apache.org/jira/browse/DERBY-2911
>             Project: Derby
>          Issue Type: Improvement
>          Components: Performance, Services
>    Affects Versions:
>            Reporter: Knut Anders Hatlen
>            Assignee: Knut Anders Hatlen
>            Priority: Minor
>         Attachments: cleaner.diff, cleaner.tar, d2911-1.diff, d2911-1.stat, d2911-2.diff,
d2911-3.diff, d2911-4.diff, d2911-5.diff, d2911-6.diff, d2911-6.stat, d2911-7.diff, d2911-7a.diff,
d2911-entry-javadoc.diff, d2911-unused.diff, d2911-unused.stat, d2911perf.java, perftest6.pdf
> There are indications that the buffer manager is a bottleneck for some types of multi-user
load. For instance, Anders Morken wrote this in a comment on DERBY-1704: "With a separate
table and index for each thread (to remove latch contention and lock waits from the equation)
we (...) found that org.apache.derby.impl.services.cache.Clock.find()/release() caused about
5 times more contention than the synchronization in LockSet.lockObject() and LockSet.unlock().
That might be an indicator of where to apply the next push".
> It would be interesting to see the scalability and performance of a buffer manager which
exploits the concurrency utilities added in Java SE 5.

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message