db-derby-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Knut Anders Hatlen (JIRA)" <j...@apache.org>
Subject [jira] Updated: (DERBY-2911) Implement a buffer manager using java.util.concurrent classes
Date Mon, 03 Dec 2007 10:34:43 GMT

     [ https://issues.apache.org/jira/browse/DERBY-2911?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Knut Anders Hatlen updated DERBY-2911:
--------------------------------------

    Attachment: d2911-9.stat
                d2911-9.diff

Attaching a new patch (d2911-9) which implements shrinking of the
clock when it has exceeded the maximum size. The shrinking is
performed in the background cleaner if the cache has one, and in the
user thread otherwise.

Also, Clock has a method called trimToSize() which tries to remove as
many invalid entries as possible (even if the cache is smaller than
the maximum size). It is called from ageOut(), discard() and
performWork(), but it doesn't do anything unless all of these
conditions are satisfied:

  1) the method must have been called at least as many times as 1/8 of
  the number of elements in the cache

  2) the cache must not contain less than 32 elements

  3) more than 25% of the elements in the cache must be invalid

This doesn't happen very often (according to the code coverage report at
http://people.apache.org/~fuzzylogic/codecoverage/529822/_files/21f.html#25
it doesn't happen at all in our test runs), so I'm not sure it adds
enough value to justify the extra complexity. However, to keep things
as close to the old implementation as possible, I added such a method
to the new buffer manager too. There is one small difference, though:
The old buffer manager would first move all invalid elements to the
end of the ArrayList, before the elements were removed from the
list. While it is possibly more efficient, this operation is performed
so infrequently that it's hardly worth the effort to optimize
it. Also, the old buffer manager would lock the entire cache while
doing this, so it knew that the elements were still invalid after it
had moved them to the end of the list. The new buffer manager only
locks one element at a time, so it doesn't know that the elements will
stay invalid after they have been moved to the end of the
list. Therefore, I chose to implement the method as a simple scan
which removed invalid entries as it found them.


I successfully ran derbyall and suites.All with the new buffer manager
enabled in modules.properties.

> Implement a buffer manager using java.util.concurrent classes
> -------------------------------------------------------------
>
>                 Key: DERBY-2911
>                 URL: https://issues.apache.org/jira/browse/DERBY-2911
>             Project: Derby
>          Issue Type: Improvement
>          Components: Performance, Services
>    Affects Versions: 10.4.0.0
>            Reporter: Knut Anders Hatlen
>            Assignee: Knut Anders Hatlen
>            Priority: Minor
>         Attachments: cleaner.diff, cleaner.tar, d2911-1.diff, d2911-1.stat, d2911-2.diff,
d2911-3.diff, d2911-4.diff, d2911-5.diff, d2911-6.diff, d2911-6.stat, d2911-7.diff, d2911-7a.diff,
d2911-9.diff, d2911-9.stat, d2911-entry-javadoc.diff, d2911-unused.diff, d2911-unused.stat,
d2911perf.java, derby-2911-8.diff, derby-2911-8.stat, perftest6.pdf, poisson_patch8.tar
>
>
> There are indications that the buffer manager is a bottleneck for some types of multi-user
load. For instance, Anders Morken wrote this in a comment on DERBY-1704: "With a separate
table and index for each thread (to remove latch contention and lock waits from the equation)
we (...) found that org.apache.derby.impl.services.cache.Clock.find()/release() caused about
5 times more contention than the synchronization in LockSet.lockObject() and LockSet.unlock().
That might be an indicator of where to apply the next push".
> It would be interesting to see the scalability and performance of a buffer manager which
exploits the concurrency utilities added in Java SE 5.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message