db-derby-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Knut Anders Hatlen (JIRA)" <j...@apache.org>
Subject [jira] Commented: (DERBY-2911) Implement a buffer manager using java.util.concurrent classes
Date Wed, 12 Sep 2007 09:13:32 GMT

    [ https://issues.apache.org/jira/browse/DERBY-2911?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12526718

Knut Anders Hatlen commented on DERBY-2911:

Some observations on the replacement algorithm in the current buffer

1) If it is known that there is at least one invalid entry in the
cache, the normal clock rotation algorithm (which scans no more than
20% of the cache) is bypassed and a scan which may go through the
entire cache is performed instead. (The rationale is that we will
avoid growing the cache if we don't have to.) This is probably OK
since invalid objects should be rare (normally present only after DROP
TABLE or similar). However, because objects that are about to be
inserted into the cache are regarded as invalid but kept so that they
can't actually be reused by someone else yet, the old buffer manager
has suffered from starting these scans too frequently (see

The new buffer manager should try to avoid this problem by only
starting a scan for invalid objects if it knows that there are items
that have been invalidated by removal.

2) When the clock is rotated and the hand sweeps over an unkept, not
recently used and dirty object, the global synchronization is dropped
and the object is cleaned. After cleaning the object, the global
synchronization is re-obtained, but then some other thread may have
moved the clock hand past the cleaned object while the first thread
didn't hold the global synchronization lock. In that case, the first
thread has cleaned an object but is not allowed to reuse its
entry. Perhaps it's just a theoretical problem, but this could mean
that some threads have to write multiple dirty pages to disk before
they are allowed to insert a new one into the cache.

Since the new buffer manager uses synchronization with finer
granularity, it should avoid this problem by keeping the
synchronization lock while the object is being cleaned. Then it knows
that the entry can be reused as soon as the object has been cleaned.

> Implement a buffer manager using java.util.concurrent classes
> -------------------------------------------------------------
>                 Key: DERBY-2911
>                 URL: https://issues.apache.org/jira/browse/DERBY-2911
>             Project: Derby
>          Issue Type: Improvement
>          Components: Performance, Services
>    Affects Versions:
>            Reporter: Knut Anders Hatlen
>            Assignee: Knut Anders Hatlen
>            Priority: Minor
>         Attachments: d2911-1.diff, d2911-1.stat, d2911-2.diff, d2911-3.diff, d2911-unused.diff,
d2911-unused.stat, d2911perf.java
> There are indications that the buffer manager is a bottleneck for some types of multi-user
load. For instance, Anders Morken wrote this in a comment on DERBY-1704: "With a separate
table and index for each thread (to remove latch contention and lock waits from the equation)
we (...) found that org.apache.derby.impl.services.cache.Clock.find()/release() caused about
5 times more contention than the synchronization in LockSet.lockObject() and LockSet.unlock().
That might be an indicator of where to apply the next push".
> It would be interesting to see the scalability and performance of a buffer manager which
exploits the concurrency utilities added in Java SE 5.

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message