db-derby-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Knut Anders Hatlen (JIRA)" <j...@apache.org>
Subject [jira] Commented: (DERBY-801) Allow parallel access to data files.
Date Thu, 02 Nov 2006 23:20:18 GMT
    [ http://issues.apache.org/jira/browse/DERBY-801?page=comments#action_12446767 ] 
Knut Anders Hatlen commented on DERBY-801:

Thanks Anders. I committed DERBY-801-6.patch with revision 470573.

> I'm not sure checking for the committed drop state in the readPage
> method is something we want to do - flushing the cache into a black
> hole is one thing, trying to read data that is supposed to be gone
> is another

You are of course right about the error checking in readPage().

> I've sort of got a feeling that maybe we are masking a problem here
> - should anyone (even the cache?) write to a dropped container?

Actually, I think it's the other way around. We are not writing to a
dropped container, but dropping a container which we happen to be
writing to. Since the dropping of the container has been committed, we
know that we'll never need those pages, so there's no need to complete
the write operations. The alternative would be to let closeContainer()
wait until all write operations on the container have finished.

> Maybe we should remove pages destined for a dying container from the
> cache when the container is dropped?

After the container has been dropped, it shouldn't be a problem
because getCommittedDropState() returns true, so writePage() will
return immediately without trying to write the page. Also, the pages
will be marked as invalid so that the cache space can be reclaimed.

> Allow parallel access to data files.
> ------------------------------------
>                 Key: DERBY-801
>                 URL: http://issues.apache.org/jira/browse/DERBY-801
>             Project: Derby
>          Issue Type: Improvement
>          Components: Performance, Store
>    Affects Versions:,,,
>         Environment: Any
>            Reporter: Øystein Grøvlen
>         Assigned To: Anders Morken
>         Attachments: DERBY-801-6.patch, DERBY-801-v2.patch, DERBY-801-v3.patch, DERBY-801-v4.patch,
DERBY-801-v5.patch, NIO-RAFContainer-v1.patch
> Derby currently serializes accesses to a data file.  For example, the
> implementation of RAFContainer.readPage is as follows:
>     synchronized (this) {  // 'this' is a FileContainer, i.e. a file object
>         fileData.seek(pageOffset);  // fileData is a RandomAccessFile
>         fileData.readFully(pageData, 0, pageSize);
>     }
> I have experiemented with a patch where I have introduced several file
> descriptors (RandomAccessFile objects) per RAFContainer.  These are
> used for reading.  The principle is that when all readers are busy, a
> readPage request will create a new reader.  (There is a maximum number
> of readers.)  With this patch, throughput was improved by 50% on
> linux.  For more discussion on this, see
> http://www.nabble.com/Derby-I-O-issues-during-checkpointing-t473523.html
> The challenge with the suggested approach is to make a mechanism to
> limit the number of open file descpriptors.  Mike Matrigali has
> suggested to use the existing CacheManager infrastructure for this
> purpose.  For a discussion on that, see:
> http://www.nabble.com/new-uses-for-basic-services-cache---looking-for-advice-t756863.html

This message is automatically generated by JIRA.
If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira


View raw message