db-derby-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Knut Anders Hatlen (JIRA)" <j...@apache.org>
Subject [jira] Updated: (DERBY-2911) Implement a buffer manager using java.util.concurrent classes
Date Fri, 16 Nov 2007 15:23:43 GMT

     [ https://issues.apache.org/jira/browse/DERBY-2911?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel

Knut Anders Hatlen updated DERBY-2911:

    Attachment: derby-2911-8.stat

Attaching a new patch (derby-2911-8.diff) which implements the
background cleaner as discussed earlier.

The patch adds a class called BackgroundCleaner. This class implements
the Serviceable interface and subscribes to the cache manager's daemon
service. When a user thread finds an object that needs to be cleaned,
it gives it to BackgroundCleaner, which cleans the object in the
daemon service's thread instead of the user thread. BackgroundCleaner
keeps the objects it needs to clean in a queue with a maximum size of
1/10 of the cache size. (There is no particular reason for choosing
exactly 1/10, it just sounded not too big and not too small. Anyway,
it's basically just an array pointing to objects already in the cache,
so it's memory footprint will be small compared to the rest of the
cache.) If the queue is full when the background cleaner receives a
request from a user thread, the user thread will have to clean the
object itself.

I also incorporated Jørgen's suggestion about releasing
synchronization lock on the entry while it is being cleaned. This also
led to a small change in how the checkpoint would write the
pages. Basically, when the checkpoint or the replacement algorithm
cleans an object, it will increase the entry's keep count (without
marking it as recently used) to prevent eviction, and release the
synchronization lock on the entry. When the sync lock has been
released, the object is written, and finally the sync lock is
re-obtained so that the keep count can be decremented.

The javadoc comment for Clock.useDaemonService() says the following
about multi-threading: "MT - synchronization provided by caller". The
interface (CacheManager) does not say anything about this, but I added
a similar comment in the interface javadoc, and used that guarantee to
avoid the need for synchronization when accessing the reference
assigned in ConcurrentCache.useDaemonService(). (Currently, this
method is only used when the database is booted, so I believe the
comment in Clock is correct.)

Derbyall and suites.All ran cleanly on Solaris 10 and Sun JDK 6 when I
manually enabled the new buffer manager in modules.properties. As for
the previous patches, this patch does not affect any code used by
default in Derby.

> Implement a buffer manager using java.util.concurrent classes
> -------------------------------------------------------------
>                 Key: DERBY-2911
>                 URL: https://issues.apache.org/jira/browse/DERBY-2911
>             Project: Derby
>          Issue Type: Improvement
>          Components: Performance, Services
>    Affects Versions:
>            Reporter: Knut Anders Hatlen
>            Assignee: Knut Anders Hatlen
>            Priority: Minor
>         Attachments: cleaner.diff, cleaner.tar, d2911-1.diff, d2911-1.stat, d2911-2.diff,
d2911-3.diff, d2911-4.diff, d2911-5.diff, d2911-6.diff, d2911-6.stat, d2911-7.diff, d2911-7a.diff,
d2911-entry-javadoc.diff, d2911-unused.diff, d2911-unused.stat, d2911perf.java, derby-2911-8.diff,
derby-2911-8.stat, perftest6.pdf
> There are indications that the buffer manager is a bottleneck for some types of multi-user
load. For instance, Anders Morken wrote this in a comment on DERBY-1704: "With a separate
table and index for each thread (to remove latch contention and lock waits from the equation)
we (...) found that org.apache.derby.impl.services.cache.Clock.find()/release() caused about
5 times more contention than the synchronization in LockSet.lockObject() and LockSet.unlock().
That might be an indicator of where to apply the next push".
> It would be interesting to see the scalability and performance of a buffer manager which
exploits the concurrency utilities added in Java SE 5.

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message