db-derby-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Anders Morken (JIRA)" <j...@apache.org>
Subject [jira] Updated: (DERBY-801) Allow parallel access to data files.
Date Wed, 01 Nov 2006 21:25:18 GMT
     [ http://issues.apache.org/jira/browse/DERBY-801?page=all ]

Anders Morken updated DERBY-801:

    Attachment: DERBY-801-v5.patch

I'm attaching a patch (DERBY-801-v5.patch (relative to trunk), which incorporates Suresh's
suggestion that I use the built-in Monitor mechanisms to boot an appropriate version of the
BaseDataFileFactory. This removed a bunch of useless code, and should fix the "Caught exception
when setting up rafContainerConstructor" errors. =)

M      java/engine/org/apache/derby/modules.properties
M      java/engine/org/apache/derby/impl/store/raw/data/BaseDataFileFactory.java
A      java/engine/org/apache/derby/impl/store/raw/data/BaseDataFileFactoryJ4.java
D      java/engine/org/apache/derby/impl/store/raw/data/RAFContainerFactory.java
M      java/engine/org/apache/derby/impl/store/build.xml

$ diffstat DERBY-801-v5.patch
 impl/store/build.xml                           |    2
 impl/store/raw/data/BaseDataFileFactory.java   |   29 +++---
 impl/store/raw/data/BaseDataFileFactoryJ4.java |   47 ++++++++++
 impl/store/raw/data/RAFContainerFactory.java   |  110 -------------------------
 modules.properties                             |    8 +
 5 files changed, 73 insertions(+), 123 deletions(-)

derbyall passed without errors - it just took me a loong while to actually get around to running
derbyall on this one. =)

I've also played with the LobLengthTest.junit test and provoked the error message, and what
I've seen is that the problem happens when the container is being closed and deleted - the
very same race condition case Knut Anders pointed out in previous comments. This happens when
one thread starts closing the container just after a write has almost finished - after the
write call to the FileChannel but before iosInProgress is decremented. So, it seems mostly
harmless. Maybe moving the committedDropState check to the start of writePage and skipping
the write altogether if it is in that state is a good approach? Any insight would be appreciated.

> Allow parallel access to data files.
> ------------------------------------
>                 Key: DERBY-801
>                 URL: http://issues.apache.org/jira/browse/DERBY-801
>             Project: Derby
>          Issue Type: Improvement
>          Components: Performance, Store
>    Affects Versions:,,,
>         Environment: Any
>            Reporter: Øystein Grøvlen
>         Assigned To: Anders Morken
>         Attachments: DERBY-801-v2.patch, DERBY-801-v3.patch, DERBY-801-v4.patch, DERBY-801-v5.patch,
> Derby currently serializes accesses to a data file.  For example, the
> implementation of RAFContainer.readPage is as follows:
>     synchronized (this) {  // 'this' is a FileContainer, i.e. a file object
>         fileData.seek(pageOffset);  // fileData is a RandomAccessFile
>         fileData.readFully(pageData, 0, pageSize);
>     }
> I have experiemented with a patch where I have introduced several file
> descriptors (RandomAccessFile objects) per RAFContainer.  These are
> used for reading.  The principle is that when all readers are busy, a
> readPage request will create a new reader.  (There is a maximum number
> of readers.)  With this patch, throughput was improved by 50% on
> linux.  For more discussion on this, see
> http://www.nabble.com/Derby-I-O-issues-during-checkpointing-t473523.html
> The challenge with the suggested approach is to make a mechanism to
> limit the number of open file descpriptors.  Mike Matrigali has
> suggested to use the existing CacheManager infrastructure for this
> purpose.  For a discussion on that, see:
> http://www.nabble.com/new-uses-for-basic-services-cache---looking-for-advice-t756863.html

This message is automatically generated by JIRA.
If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira


View raw message