db-derby-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Knut Anders Hatlen (JIRA)" <j...@apache.org>
Subject [jira] Commented: (DERBY-2911) Implement a buffer manager using java.util.concurrent classes
Date Mon, 22 Oct 2007 15:06:51 GMT

    [ https://issues.apache.org/jira/browse/DERBY-2911?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12536700

Knut Anders Hatlen commented on DERBY-2911:

I have not implemented a background cleaner for the new buffer manager yet. I will try to
implement one that resembles the existing one as much as possible. My understanding is that
the background cleaner can be invoked either to clean dirty pages in front of the clock hand,
or to shrink the cache if it has grown bigger than its max size. The cleaner is invoked by
rotateClock() when it detects that a page needs to be clean (but rotateClock() will clean
that page itself), or when it detects that the cache is too large. It then scans the cache,
starting right in front of the clock hand, until it finds a page to clean. During the scan,
it evicts clean and not recently used pages if the cache needs to shrink. The scan stops when
at least one of the following conditions are satisfied:

  a) a page has been cleaned
  b) it has seen that 5% of the pages in the cache are clean
  c) it has visited 10% of the pages in the cache

I'm not sure whether the restriction imposed by condition a, that the background cleaner never
cleans more than one page per invocation, is intended or not. Before I started reading the
code, I expected the background cleaner to be more aggressive and clean at least a couple
of pages before it went back to sleep. In fact, it looks like the intention at some point
was that it should requeue itself after writing out the page, as seen by this return statement
in Clock.performWork(boolean):

		needService = true;
		return Serviceable.REQUEUE; // return is actually ignored.

The reason why the return value is ignored, is that BasicDaemon never requeues a request when
the Serviceable is a subscriber. This sounds reasonable, since a subscriber would normally
be serviced again later anyway. The background cleaner is however subscribed in onDemandOnly
mode, so it won't be serviced until it puts a new request in the queue itself.

Does anyone have any details on how the background cleaner is intended to work?

> Implement a buffer manager using java.util.concurrent classes
> -------------------------------------------------------------
>                 Key: DERBY-2911
>                 URL: https://issues.apache.org/jira/browse/DERBY-2911
>             Project: Derby
>          Issue Type: Improvement
>          Components: Performance, Services
>    Affects Versions:
>            Reporter: Knut Anders Hatlen
>            Assignee: Knut Anders Hatlen
>            Priority: Minor
>         Attachments: d2911-1.diff, d2911-1.stat, d2911-2.diff, d2911-3.diff, d2911-4.diff,
d2911-5.diff, d2911-6.diff, d2911-6.stat, d2911-7.diff, d2911-7a.diff, d2911-entry-javadoc.diff,
d2911-unused.diff, d2911-unused.stat, d2911perf.java, perftest6.pdf
> There are indications that the buffer manager is a bottleneck for some types of multi-user
load. For instance, Anders Morken wrote this in a comment on DERBY-1704: "With a separate
table and index for each thread (to remove latch contention and lock waits from the equation)
we (...) found that org.apache.derby.impl.services.cache.Clock.find()/release() caused about
5 times more contention than the synchronization in LockSet.lockObject() and LockSet.unlock().
That might be an indicator of where to apply the next push".
> It would be interesting to see the scalability and performance of a buffer manager which
exploits the concurrency utilities added in Java SE 5.

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message