db-derby-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Øystein Grøvlen (JIRA) <derby-...@db.apache.org>
Subject [jira] Commented: (DERBY-801) Allow parallel access to data files.
Date Tue, 30 May 2006 14:46:30 GMT
    [ http://issues.apache.org/jira/browse/DERBY-801?page=comments#action_12413841 ] 

Øystein Grøvlen commented on DERBY-801:
---------------------------------------

I have run our tpc-b like benchmark on with and without your patch on a 2-CPU opteron box
with 2 local disks running Linux. 

Results:

Without patch:  Throughput: 70 tps.  Max response time: 47.8 seconds
With patch: Throughput: 106 tps. Max response time: 4.5 seconds

All tests were run on the same database (17.5 GB) with a 500 MB page cache.   For each code
version I ran 2 tests each lasting 1 hour.

I think this looks very good.  It increases throughput with 50% and reduces the max response
time with 90%.  I think this is comparable to the results I saw with multiple file descriptors
per file.  However, your solution is much cleaner.  Do think much work is remaining on this
patch to make it ready for production?



> Allow parallel access to data files.
> ------------------------------------
>
>          Key: DERBY-801
>          URL: http://issues.apache.org/jira/browse/DERBY-801
>      Project: Derby
>         Type: Improvement

>   Components: Performance, Store
>     Versions: 10.0.2.0, 10.0.2.1, 10.1.1.0, 10.1.1.1, 10.1.1.2, 10.1.2.0, 10.1.2.1
>  Environment: Any
>     Reporter: Øystein Grøvlen
>  Attachments: NIO-RAFContainer-v1.patch
>
> Derby currently serializes accesses to a data file.  For example, the
> implementation of RAFContainer.readPage is as follows:
>     synchronized (this) {  // 'this' is a FileContainer, i.e. a file object
>         fileData.seek(pageOffset);  // fileData is a RandomAccessFile
>         fileData.readFully(pageData, 0, pageSize);
>     }
> I have experiemented with a patch where I have introduced several file
> descriptors (RandomAccessFile objects) per RAFContainer.  These are
> used for reading.  The principle is that when all readers are busy, a
> readPage request will create a new reader.  (There is a maximum number
> of readers.)  With this patch, throughput was improved by 50% on
> linux.  For more discussion on this, see
> http://www.nabble.com/Derby-I-O-issues-during-checkpointing-t473523.html
> The challenge with the suggested approach is to make a mechanism to
> limit the number of open file descpriptors.  Mike Matrigali has
> suggested to use the existing CacheManager infrastructure for this
> purpose.  For a discussion on that, see:
> http://www.nabble.com/new-uses-for-basic-services-cache---looking-for-advice-t756863.html

-- 
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
   http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see:
   http://www.atlassian.com/software/jira


Mime
View raw message