db-derby-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Mike Matrigali <mikem_...@sbcglobal.net>
Subject Re: [jira] Commented: (DERBY-239) Need a online backup feature that does not block update operations when online backup is in progress.
Date Thu, 04 Aug 2005 23:09:58 GMT
I don't think I followed all the checkpoint issues, but while
thinking about it I had these thoughts:

o there are at least 2 reasons to have checkpoints:
  1) allow garbage collection of log file space, if logs are
     not required for rollfoward backup recovery.
  2) to minimize the amount of redo work necessary if
     a crash occurs.

With respect to option #2 I think you have have to be careful in
online backup, I think in the backup you need to set the redo
low water mark to a checkpoint that predates the start of the
backup, you can not use a subsequent checkpoint that happens
during the backup.  This is because you are just going to copy
data pages in order and the backup copy may have a page version
before those represented in a checkpoint.

Suresh Thalamati wrote:

> Mike Matrigali wrote:
> 
>> o I think it would seem reasonable to leave decision of whether to wait
>>   for unlogged operations to finish up to the caller of the backup.
>>   For some set of users waiting is fine, they just want the backup
>>   eventually, and they are using the feature because they want normal
>>   users to be impacted as little as possible (at least with respect to
>>   lock concurrency).  Others may need to know imediately why something
>>   is blocking their operation.
>>
> 
> That is a good point, but to provide both the options; new backup
> procedures that takes an extra parameter for the users to specify
> wait/fail are required.
> 
> I am thinking of providing following two additional backup procedures
> for this case:
> 
> 1) Full backup
> SYSCS_UTIL.SYSCS_BACKUP_DATABASE(IN BACKUPDIR VARCHAR(),
>           IN WAIT_FOR_NONLOGGED_OPERATIONS SMALLINT)
> 
> and
> 
> 2) Full backup with log archive mode:
> 
> SYSCS_UTIL.SYSCS_BACKUP_DATABASE_AND_ENABLE_LOG_ARCHIVE_MODE
> (IN BACKUPDIR VARCHAR(32672), IN SMALLINT DELETE_ARCHIVED_LOG_FILES,
>           IN WAIT_FOR_NONLOGGED_OPERATIONS SMALLINT))
> 
> 
> WAIT_FOR_NONLOGGED_OPERATIONS :
> A non-zero input argument of type SMALLINT will make backup procedures
> to wait for the non-logged operation that are currently running to
> commit, while an argument of 0 will make backup fail if there are any
> uncommitted non-logged operations.
> 
> Please note that the signature of the these procedures is same as
> the current procedures , except for the additional argument.
> 
> Current procedures has to be maintained for backward compatibility
> reasons. They will work in non-blocking mode(I doubt any user will
> complain about it!). Existing backup procedures will wait for non-logged
> operations commit.
> 
> Any comments ?
> 
> 
>> o I had just assumed that online backup would only work in roll foward
>>   mode, so there was really no garbage collection anyway - at least not
>>   until we provide a way to move rollforward logs somewhere
>> automatically.  I think this is sort of along the lines of what Oystein
>> is saying, would be good to do in future but 1st impl does not need to
>> address.  Remember while Derby does not have built in size limits, it
>> does not have features one expects from VLDB (for really large db's you
>> probably want to have multiple threads/devices backup up in parallel
>> along with simultaneous log file moves).
>>
> 
> Mike, I think it will be more helpful to the users if Derby supports
> non-blocking simple full backups, not just non-blocking backups only
> with log archive mode enabled.
> 
> I believe Øystein is referring to the non lograchive mode scenario, if
> there are checkpoints when backup is in progress; log space can released
> by copying the log files to the backup in parallel to the data files
> copy operation. I also think, this enhancement is not essential for 
> first implementation of real time online backup.
> 
> 
> Thanks
> -suresht
> 
> 
> 
>>
>>
>> Øystein Grøvlen wrote:
>>
>>
>>>>>>>> "ØG" == Øystein Grøvlen <Oystein.Grovlen@Sun.COM>
writes:
>>>
>>>
>>>
>>>>>>>> "ST" == Suresh Thalamati <suresh.thalamati@gmail.com>
writes:
>>>
>>>
>>>    ST> My  initial thought  is to  simply disable 
>>> garbage-collection  of log
>>>    ST> files for the  duration of the backup. unless  there are some
>>> specific
>>>    ST> advantages in writing backup-start log record.
>>>
>>>    ØG> Disabling garabage-collection directly is probably the
>>> cleanest way to
>>>    ØG> do this.
>>>
>>> On second thoughts, for large databases, the time to copy the data
>>> files may be so long that the log volume becomes prohibitively large
>>> if garbage collection is turned off for the entire backup.  In that
>>> case, it would be better to copy finished log files in parallel with
>>> the data files and let them be garbage collected at any time after it
>>> has been copied.  This probably does not need to be part of a first
>>> implementation, but something one should consider for the future.
>>>
>>
>>
> 
> 
> 

Mime
View raw message