db-derby-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Dibyendu Majumdar <dibye...@mazumdar.demon.co.uk>
Subject Re: Discussion of incremental checkpointing
Date Wed, 15 Feb 2006 00:41:05 GMT
Hi,

Mohan's paper on ARIES describes a way to perform lightweight 
checkpoints. The basic idea is:

+ Checkpoints don't cause data pages to be flushed. Instead, a list of 
dirty pages is recorded in the checkpoint. Along with the id of each 
dirty page, its Recovery LSN is stored, which is the LSN of the oldest 
log record that may have made changes to the page.
+ Dirty pages are flushed separately using a Buffer Writer.

Derby at present implements the fuzzy checkpoint algorithm described in 
section 11.3 of TPCT book.

For anyone interested in a pure ARIES implementation - please have a 
look at my project - www.simpledbm.org.

I guess the main difference between ARIES and Derby's method is that in 
ARIES, checkpoints happen independently of Buffer Writes. However, in 
the end the amount of work done is the same, as the data pages must be 
written out anyway.

I think I agree with Mike's comment that we need to be clear about what 
it is that we are trying to solve. Are you trying to:

1. Make checkpoints lightweight like ARIES? If so, this would require 
major changes to recovery logic.
2. Optimize the IO during checkpoints? I guess if this is the goal, then 
one should concentrate on improving the current IO logic - potentially 
by ensuring that:
a) IO is non-disruptive - ie  - pages are not locked out while they are 
being written.
b) IO is optimized - batch write consecutive pages, etc.
c) Have multiple concurrent threads handle IO.
However these changes are useful in large scale environments and will 
actually have a negative impact on a small scale system. No point in 
increasing IO if there is a single disk, for example.
3. Slow down the IO during checkpoints? I think this is what Mike wants 
to do. As far as I know, increasing the time taken to perform a 
checkpoint should have no adverse effect. For example, you could reduce 
the sleep time between checkpoints but slow down the checkpoint itself. 
This way you could keep the total number of pages written fairly 
constant without causing peeks of IO.

Regards

Dibyendu


Mime
View raw message