db-derby-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Suresh Thalamati (JIRA)" <derby-...@db.apache.org>
Subject [jira] Commented: (DERBY-801) Allow parallel access to data files.
Date Thu, 14 Sep 2006 08:43:25 GMT
    [ http://issues.apache.org/jira/browse/DERBY-801?page=comments#action_12434613 ] 
Suresh Thalamati commented on DERBY-801:

Thanks for working on this issue, Anders. I really like your solution to solve
this issue. Patch is very good, I have only few minor comments/questions.
 I am really sorry  for not reviewing it sooner. 


Logic in this new class seems to be deciding whether to load RafContainer.java or
the RafContainer4.java based on the JVM.  I am not sure, if  this logic is
necessary here. Did you consider using basic services to load the java classes 
specific to a JVM ?

I think basic services has support to boot a specific factory implementation
based on the JVM using modules.properties. For example in the current
scenario, one can extend BaseDataFileFactory.java class to  
implement newContainerObject(), which will return the RafContainer4( ..).  add
the new class to modules.properties to boot only on versions >=jdk14.

In RafContainer4.java :

1) I think following import is not needed. 

+import java.io.*; 

2) Is it really necessary to rewind() the buffers in readFull/writeFull ? From what I understood,

   there is a new ByteBuffer object being created on  both read/write page

+        dstBuffer.rewind(); // Reset buffer position before we start read
+        srcBuffer.rewind(); // Reset buffer position before we start writing.

3) do we really need the following method ? 

+    final protected FileChannel getChannel() {
+        return ourChannel;
+    }

4) I noticed, there is new encryption buffer created on every writePage() call,
   if the database is encrypted. This may cause jvm peak memory usage increase,
   when there is a checkpoint, if there are lot of dirty pages in the cache and
   if garbage collection is not happening fast enough.  I hope this does not
   lead to out of memory errors!

   We may need to implement some kind of scheme, that will help in reuse of
   the encryption buffers. 
5) I am ok with readPage() and writePage() routines in RafContainer4.java. 
   just curious , if you considered implementing  new read/write..etc  calls in 
   the RafContainer4.java using file channel and just wrapper methods in the
   RafContainer.java using the existing random access file,  instead of 
   overriding readPage()/writePage() ...etc. 

6) Please file a JIRA  to enhance StorageFactory interfaces to support NIO. 


> Allow parallel access to data files.
> ------------------------------------
>                 Key: DERBY-801
>                 URL: http://issues.apache.org/jira/browse/DERBY-801
>             Project: Derby
>          Issue Type: Improvement
>          Components: Performance, Store
>    Affects Versions:,,,,,,
>         Environment: Any
>            Reporter: Øystein Grøvlen
>         Assigned To: Anders Morken
>         Attachments: DERBY-801-v2.patch, DERBY-801-v3.patch, DERBY-801-v4.patch, NIO-RAFContainer-v1.patch
> Derby currently serializes accesses to a data file.  For example, the
> implementation of RAFContainer.readPage is as follows:
>     synchronized (this) {  // 'this' is a FileContainer, i.e. a file object
>         fileData.seek(pageOffset);  // fileData is a RandomAccessFile
>         fileData.readFully(pageData, 0, pageSize);
>     }
> I have experiemented with a patch where I have introduced several file
> descriptors (RandomAccessFile objects) per RAFContainer.  These are
> used for reading.  The principle is that when all readers are busy, a
> readPage request will create a new reader.  (There is a maximum number
> of readers.)  With this patch, throughput was improved by 50% on
> linux.  For more discussion on this, see
> http://www.nabble.com/Derby-I-O-issues-during-checkpointing-t473523.html
> The challenge with the suggested approach is to make a mechanism to
> limit the number of open file descpriptors.  Mike Matrigali has
> suggested to use the existing CacheManager infrastructure for this
> purpose.  For a discussion on that, see:
> http://www.nabble.com/new-uses-for-basic-services-cache---looking-for-advice-t756863.html

This message is automatically generated by JIRA.
If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira


View raw message