db-derby-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Suresh Thalamati <suresh.thalam...@gmail.com>
Subject Re: Options for syncing of log to disk
Date Wed, 30 Aug 2006 19:16:47 GMT
Olav Sandstaa wrote:

> 
> I have run some simple tests where I have changed mode from "rws" to
> "rwd" for the Derby log file. When running a small numbers of
> concurrent client threads the throughput is almost doubled and the
> response time is almost halved. I am enclosing two graphs that show
> this when running a given number of concurrent "tpc-b" clients. The
> graphs show the throughput when running with "rws" and "rwd" mode when the
> disk's write cache has been enabled and disabled.

I am really surprised to notice the difference between rws / rwd 
throughput in your graphs,  Especially when the write cache is 
enabled.  When I ran small tests long time ago, did not notice any 
substantial difference between rws/rwd, on windows for sure.

Could u please post more details on the hardware/OS/JVM ..etc.  and 
What is the work done in each transaction ?


> 
> This change should also have a positive impact on the Derby startup
> time (DERBY-1664) and derbyall. With this change the time for running
> derbyall goes down by about 10-15 minutes (approximately 10%) :-)


Another interesting thing to explore here might be to, is it really 
necessary to flush the log for the create tables during database 
creation.  Some time ago, when I dumped the log for the create 
database, only thing that I noticed was create table log records; Most 
of the other action are done in Un-logged mode during creation. Just 
for the record, these log flush is not specific to write sync mode, 
they use to happen when the log was synced using file sync.

I was almost tempted to disable logging for those creates, but then 
thought why add any extra code for the regular create table path for
database creation special case, which is suppose to happen only once :-)



> 
> Is there anyone that is aware of any issues by not updating the file
> meta-data for every write? Is there any recovery scenarios where this
> can make recovery fail? Derby seems to preallocates the log file
> before starting using the file, so I think this should not influence
> the ability to fine the last data written to the file after a power
> failure.
> 


Log is not preallocated dynamically ; it happens only on a log file 
switch.  Log file switch is triggered during log flush calls 
(commit/abort). If a transaction is long one, it is possible to write 
  the log records to non-preallocation portion of a log file.

In non-preallocated log file scenario, If a meta data(Never managed to 
figure out what that exactly means in various operating systems) is 
not updated, last data written to the log file might be lost.



Thanks
-suresht



Mime
View raw message