db-derby-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Mike Matrigali <mikem_...@sbcglobal.net>
Subject Re: Discussion of how to map the recovery time into Xmb of log --Checkpoint issue
Date Wed, 25 Jan 2006 23:19:40 GMT

Raymond Raymond wrote:
> Mike, last time you gave me some comments about how to map the
> recovery time into Xmb of log. I still have some question about it.
> RR    2. During initilization of Derby, we run some measurement that
> RR       determines the performance of the system and maps the
> RR       recovery time into some X megabytes of log.
> RR  Now, I am going to design the 2nd step first to map the recovery
> RR  time into some
> RR  X megabytes of log. A simple approach is that we can design a test
> RR  log file. In the
> RR  log file, we can let derby create a temporary database and do a
> RR  bunch of test to get
> RR  necessary disk IO information, and then delete the temporary
> RR  database. When derby
> RR  boots up, we let it to do recovery from the test log file.Anyone
> RR  has some other
> RR  suggestions on it?
> MM  I'll think about this, it is not straight forward.  My guess would
> MM  be that recovery time is dominated by 2 factors:
> MM  1) I/O from log disk
> MM  2) I/O from data disk
> MM
> MM  Item 1 is pretty easy to get a handle on.  During redo it is pretty
> MM  much
> MM  a straight scan from beginning to end doing page based I/O.  Undo is
> MM  harder as it jumps back and forth for each xact.  I would probably
> MM  just
> MM  ignore it for estimates.
> MM
> MM  Item 2 is totally dependent on cache rate hit you are going to
> MM  expect, and the number of log records.
> MM  The majority of log records deal with a single page, it will read
> MM  the page into cache if it doesn't exist and then it will do a quick
> MM  operation on that page.  Again undo is slightly more complicated as
> MM  it
> MM  could involve logical lookups in the index.
> MM
> MM  Another option rather than do any sort of testing is to come up with
> MM  an
> MM  initial default time based on size of log file.  And then on each
> MM  subsequent recovery event dynamically change the estimate based on
> MM  how
> MM  long that recovery on that db took.  This way each estimate will be
> MM  based on the actual work generated by the application, and over time
> MM  should become better and better.
> I agree with you it is better to estimate with actual work generated by
> the appplication. But, as I know, derby only performs recovery when it
> boots(am I right here?). If Derby runs stably, the "subsequent recovery
> event" you mentioned will not happen and we can't get the information we
> need.
> RR
> RR  Raymond
> _________________________________________________________________
> Take advantage of powerful junk e-mail filters built on patented 
> Microsoft® SmartScreen Technology. 
> http://join.msn.com/?pgmarket=en-ca&page=byoa/prem&xAPID=1994&DI=1034&SU=http://hotmail.com/enca&HL=Market_MSNIS_Taglines

>  Start enjoying all the benefits of MSN® Premium right now and get the 
> first two months FREE*.

Yes we run recovery at boot time, and if Derby is always shutdown 
cleanly then there won't be much work to do.  In that case the user
should not care much that we don't have a good estimate as it seems
on that system we never do recovery.  The downside is that the first
time they do recovery we may not meet the expected time of recovery
window we were aiming at.

Note we could refine the estimate from the running system by a little
by gathering average log record size.  This would give an estimate
of number of possible I/O's needed to do the recovery - maybe that
along with a simple current cache hit stat that is maintained in the
system could be used.

For me it would seem ok in a zero admin db, to make a good guess at
the recovery time based on some static assumptions and then refine
the guess over time as needed using subsequent recovery events.  I
really don't like the idea of slowing down every boot of the database
with a statistics gathering phase.  And even slowing down every
create of the database is going to annoy some people as we have had
some users where create database time is key to their app (I believe
it was an apache project using a separate db for each test they were

In a non-zero admin db I would say go ahead and implement the tests, and 
then provide a way for the dba to schedule the statistics gathering.

View raw message