db-derby-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Anders Morken (JIRA)" <j...@apache.org>
Subject [jira] Resolved: (DERBY-801) Allow parallel access to data files.
Date Wed, 15 Nov 2006 15:56:42 GMT
     [ http://issues.apache.org/jira/browse/DERBY-801?page=all ]

Anders Morken resolved DERBY-801.

    Resolution: Fixed

OK, I'm marking this one as resolved, then. I've logged DERBY-2086 to remind us to look into
some kind of resource pooling mechanism for encryption buffers and similar. I guess this isn't
the only place where a pooling mechanism could be useful, so I intentionally worded it in
a generic fashion.

Regarding enhancing the StorageFactory interfaces to support general parallel access, I'm
not sure I understand Suresh's intention completely. Implementing support for concurrent IO
is specific to the different container types - and it may not even be feasible for compressed
containers that require on decompression for seeks. Suresh, maybe you could explain it in
a new Jira issue? ;-)

As I considered earlier, we could enhance the StorageRandomAccessFile interface to add getChannel()
support, but even that would apply only to random access files backed by real files that do
support getChannel - and we'd have to throw in some JVM-dependent trickery as well, since
getChannel() in RandomAccessFile only exists in Java 1.4+. (What's the timeframe on deprecating
1.3 support? I guess it could be a while? =)

Anyway, thanks to everyone who helped me on this one. =)

> Allow parallel access to data files.
> ------------------------------------
>                 Key: DERBY-801
>                 URL: http://issues.apache.org/jira/browse/DERBY-801
>             Project: Derby
>          Issue Type: Improvement
>          Components: Performance, Store
>    Affects Versions:,,,
>         Environment: Any
>            Reporter: Øystein Grøvlen
>         Assigned To: Anders Morken
>             Fix For:
>         Attachments: DERBY-801-6.patch, DERBY-801-7.patch, DERBY-801-v2.patch, DERBY-801-v3.patch,
DERBY-801-v4.patch, DERBY-801-v5.patch, NIO-RAFContainer-v1.patch
> Derby currently serializes accesses to a data file.  For example, the
> implementation of RAFContainer.readPage is as follows:
>     synchronized (this) {  // 'this' is a FileContainer, i.e. a file object
>         fileData.seek(pageOffset);  // fileData is a RandomAccessFile
>         fileData.readFully(pageData, 0, pageSize);
>     }
> I have experiemented with a patch where I have introduced several file
> descriptors (RandomAccessFile objects) per RAFContainer.  These are
> used for reading.  The principle is that when all readers are busy, a
> readPage request will create a new reader.  (There is a maximum number
> of readers.)  With this patch, throughput was improved by 50% on
> linux.  For more discussion on this, see
> http://www.nabble.com/Derby-I-O-issues-during-checkpointing-t473523.html
> The challenge with the suggested approach is to make a mechanism to
> limit the number of open file descpriptors.  Mike Matrigali has
> suggested to use the existing CacheManager infrastructure for this
> purpose.  For a discussion on that, see:
> http://www.nabble.com/new-uses-for-basic-services-cache---looking-for-advice-t756863.html

This message is automatically generated by JIRA.
If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira


View raw message