db-derby-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Philip Bohannon" <bohan...@lucent.com>
Subject RE: Derby architecture/design documents
Date Wed, 02 Feb 2005 00:18:16 GMT
Sounds good.  Small comments below.

Philip Bohannon
Computing Sciences Research Center
Lucent Technologies -- Bell Laboratories

> -----Original Message-----
> From: Mike Matrigali [mailto:mikem_app@sbcglobal.net]
> Sent: Tuesday, February 01, 2005 6:45 PM
> To: Derby Development
> Subject: Re: Derby architecture/design documents
> I think the only hope hear is to define the terms and then document
> what derby does in reference to them.  Philip's definitions seem
> reasonable.  Here is what derby supports in those terms.
> 1) Derby fully supports crash recovery, it uses java to correctly
>    sync the log file to support this.
> 2) I would say derby supports media recovery.  One can make a backup of
>    the data and store it off line.  Logs can be stored on a separate
>    disk from the data, and if you lose your data disk then you can use
>    rollforward recovery on the existing logs and the copy of the backup
>    to bring your database up to the current point in time.

One issue here is to easily be able to tell from a copy of a
checkpoint/backup what logs are needed to restore it.

Also an issue we ran into (in the database that we built once-upon-a-time)
is that old checkpoints and media recovery may mean "more logs than will
fit on your disk" have to be run. Thus recovery needs to be friendly and
say something like "I need logXXXXXXX, please tell me where it is?", and
wait for it to be mounted, untar'd etc. (Quickly glanced through docs and
didn't see mention of these small, but sometimes practical, points.)

> 3) Derby does not support "point in time recovery".  Someone may want to
>    look at this in the future.  Technically I don't think it would be
>    very hard as the logging system has the stuff to solve the hard
>    problems.  It does not have an idea about "time" - it just knows log
>    sequence numbers, so need to figure out what kind of interface a user
>    really wants.  A very user unfriendly interface would not be very
>    hard to implement which would be recover to a specific log sequence
>    number.  Anyone interested in this feature should add it to jira -
>    I'll be happy to add technical comments on what needs to be done.

I believe we added a real-time stamp in the begin transaction record, a
command line parameter to recovery:

recover -timestamp XXXX

and a short 'if' statement in the main recovery loop.

The feature tests, of course, were a lot more trouble :-/.


If the user has access to some stable transaction identifier, it may be
helpful to let the user recover to the commit point of a known
transaction.  This can help when the DB is cooperating with other
processes, but not running XA for whatever reason.

(I may try to create a jira entry, but if someone beats me to it, no

> 4) A reasonable next step in derby recovery progress would be to add a
>    way to automatically move/copy log files offline as they are not
>    needed by crash recovery and only needed for media recovery.  Some
>    sort of java stored procedure callout would seem most appropriate.
> /mikem
> Philip Bohannon wrote:
> > I believe "crash recovery" means all volatile buffers and non-flushed
> > pages are lost, as well as, depending on your failure model, the total
> > contents of any physical disk page in the process of being written at
> > time of the crash.
> >
> > "Media recovery" means that we lost some part of the checkpoint, but
> > have archived versions of the log, and some offline copy of an old
> > checkpoint.
> >
> > If you lose your logs (and there is a crash), AFAIK, there is no hope
> > recover information past the first loss, assuming it is after the most
> > recent checkpoint.
> >
> > Also a desirable feature is "point in time recovery" against logical
> > corruption - i.e., some angry/confused user started deleting all the
> > customer records Monday morning.  Can I start with an old checkpoint
> > run the recovery log until Sunday night to get some consistent state
> > around that time back?  (This is usually straightforward to
> >
> > Cheers,
> >
> > Philip Bohannon
> > Computing Sciences Research Center
> > Lucent Technologies -- Bell Laboratories
> >
> >
> >>-----Original Message-----
> >>From: Dibyendu Majumdar [mailto:dibyendu@mazumdar.demon.co.uk]
> >>Sent: Tuesday, February 01, 2005 2:43 PM
> >>To: Derby Development
> >>Subject: Re: Derby architecture/design documents
> >>
> >>
> >>From: "Suresh Thalamati" <suresh.thalamati@gmail.com>
> >>
> >>> >>Derby implements the Write Ahead Log using a non-circular file
> >
> > system
> >
> >>>file. At present, there is no support for incremental log backup or
> >>>media recovery. Only crash recovery is supported.
> >>>
> >>>I think derby does support simple media recovery.  It has support for
> >>>full backup/restore and very basic form of  rollforward recovery
> >
> > (replay
> >
> >>>of logs using backup and archived log files).
> >>
> >>Hi Suresh,
> >>
> >>Thanks for the feedback. I suppose that we should define media
> >
> > and
> >
> >>crash recovery. My understanding is that media recovery is when you
> >>lost your logs as
> >>well, whereas crash recovery is when the logs are intact. In case of
> >
> > media
> >
> >>recovery,
> >>does Derby know how to locate the last checkpoint record/log file?
> >>
> >>Regards
> >>
> >>
> >>

View raw message