db-derby-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Anders Morken (JIRA)" <derby-...@db.apache.org>
Subject [jira] Updated: (DERBY-801) Allow parallel access to data files.
Date Tue, 12 Sep 2006 21:32:23 GMT
     [ http://issues.apache.org/jira/browse/DERBY-801?page=all ]

Anders Morken updated DERBY-801:

    Attachment: DERBY-801-v4.patch

Thanks again for your review and comments, Knut Anders, I appreciate it. Sorry for the long
delay, but I've gotten busy with my autumn project - incidentally it's related to your work
on DERBY-1704. =)

Now, you've convinced me regarding updatePageArray, I've removed it from the patch. =)

Regarding writePage() while the container is in the committedDrop state, I think it can happen
in two cases: Somebody's writing to the container without obeying the lock protocols in Derby
- or the cache is cleaning out dirty pages it is holding to the container.

According to  BaseDataFileFactory#dropContainer(), the Container is locked in container exclusive
mode while it is being closed and dropped. I haven't investigated the cache to see if it checks
any such locks, but I presume it relies on it's clients to have the appropriate locks before
writing to the container through the cache. 

Since the original check for the commited drop state just makes writes to a committed dropped
container a noop, I figure we can do that if somebody races in and closes the container between
the first check and the actual write as well - so I've added a little exception handling which
hopefully does the Right Thing. Comments or better suggestions welcome, of course.

So, here we go, DERBY-801-v4.patch. Hit it. Both storeall and encryptionAll passed before
I added the exception handling mentioned above, and storeall passed just great on the attached
patch as well. =)

> Allow parallel access to data files.
> ------------------------------------
>                 Key: DERBY-801
>                 URL: http://issues.apache.org/jira/browse/DERBY-801
>             Project: Derby
>          Issue Type: Improvement
>          Components: Performance, Store
>    Affects Versions:,,,,,,
>         Environment: Any
>            Reporter: Øystein Grøvlen
>         Assigned To: Anders Morken
>         Attachments: DERBY-801-v2.patch, DERBY-801-v3.patch, DERBY-801-v4.patch, NIO-RAFContainer-v1.patch
> Derby currently serializes accesses to a data file.  For example, the
> implementation of RAFContainer.readPage is as follows:
>     synchronized (this) {  // 'this' is a FileContainer, i.e. a file object
>         fileData.seek(pageOffset);  // fileData is a RandomAccessFile
>         fileData.readFully(pageData, 0, pageSize);
>     }
> I have experiemented with a patch where I have introduced several file
> descriptors (RandomAccessFile objects) per RAFContainer.  These are
> used for reading.  The principle is that when all readers are busy, a
> readPage request will create a new reader.  (There is a maximum number
> of readers.)  With this patch, throughput was improved by 50% on
> linux.  For more discussion on this, see
> http://www.nabble.com/Derby-I-O-issues-during-checkpointing-t473523.html
> The challenge with the suggested approach is to make a mechanism to
> limit the number of open file descpriptors.  Mike Matrigali has
> suggested to use the existing CacheManager infrastructure for this
> purpose.  For a discussion on that, see:
> http://www.nabble.com/new-uses-for-basic-services-cache---looking-for-advice-t756863.html

This message is automatically generated by JIRA.
If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira


View raw message