db-derby-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Øystein Grøvlen <Oystein.Grov...@Sun.COM>
Subject Re: Derby I/O issues during checkpointing
Date Mon, 06 Mar 2006 21:13:27 GMT
Raymond Raymond wrote:
> Dear Oystein,
> In your mail, "Derby I/O issues during checkpointing", you wrote:
> OO:  Some tests runs we have done show very long transaction response times
> OO:  during checkpointing.  This has been seen on several platforms.  The
> OO:  load is TPC-B like transactions and the write cache is turned off so
> OO:  the system is I/O bound.  There seems to be two major issues:
> OO:
> OO:  1. Derby does checkpointing by writing all dirty pages by
> OO:      RandomAccessFile.write() and then do file sync when the entire
> OO:      cache has been scanned.  When the page cache is large, the file
> OO:      system buffer will overflow during checkpointing, and occasionally
> OO:      the writes will take very long.  I have observed single write
> OO:      operations that took almost 12 seconds.  What is even worse is 
> that
> OO:      during this period also read performance on other files can be 
> very
> OO:      bad.  For example, reading an index page from disk can take close
> OO:      to 10 seconds when the base table is checkpointed.  Hence,
> OO:      transactions are severely slowed down.
> OO:
> OO:      I have managed to improve response times by flushing every file 
> for
> OO:      every 100th write.  Is this something we should consider including
> OO:      in the code?  Do you have better suggestions?
> Has this been implemented in the latest derby version?

No, not yet.  I wanted to look at some other options first.

> I am trying to 
> spread
> out disk I/O of checkpoint over the checkpoint interval. If the 
> RandomAccessFile
> still does sync when the entire cache has been scanned, my approach will 
> not
> make much difference from current implementation. 

I do not understand why syncing only once should affect your approach 
more than syncing many times.

> If it has not been 
> implemented
> in the latest derby version, would you please attach your solution? I would
> like to use    it in my working copy.

I do not have access to it now.  I can send it tomorrow.  Basically, I 
incremented a counter on every write  in RAFContainer.  When the counter 
reached 100, I did a sync and reset the counter.

> I am also curious to know, in current implementation, when will the sync
> of a RandomAccessFile be executed?

I think Mike already answered this.


View raw message