Mailing-List: contact derby-dev-help@db.apache.org; run by ezmlm
Precedence: bulk
Reply-To: "Derby Development" <derby-dev@db.apache.org>
Message-ID: <1570751357.1121993566296.JavaMail.jira@ajax.apache.org>
Date: Fri, 22 Jul 2005 02:52:46 +0200 (CEST)
From: "Suresh Thalamati (JIRA)" <derby-dev@db.apache.org>
To: derby-dev@db.apache.org
Subject: [jira] Commented: (DERBY-239) Need a online backup feature  that does
 not block update operations   when online backup is in progress.
In-Reply-To: <131892714.1114538125243.JavaMail.jira@ajax.apache.org>
Mime-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit

    [ http://issues.apache.org/jira/browse/DERBY-239?page=comments#action_12316434 ] 

Suresh Thalamati commented on DERBY-239:
----------------------------------------

I think providing  an online backup mechanism that does not block changes
to the database when the backup is in progress will be a useful feature to Derby users,
especially in the client/server environment. This backup mechanism might take more time
than current online backup because of the synchronization overheads required to  allow changes to
the database when backup is in progress. At this point I am not sure how much more
time it will take, but I think it should not be more than 50%, in the worst case scenario. 

Current online backup mechanism (that blocks changes to the database) is
supported using system procedures(ex:SYSCS_UTIL.SYSCS_BACKUP_DATABASE ). My
plan is to make the existing backup procedures work work without blocking the
changes to the database; No new system procedures are required. If community thinks 
both blocking/non-blocking type backups are useful, new procedures can 
be added. 

Currently backup contains mainly data files (seg0/*) and the transaction log
files(log/*) that are there when the backup started. On restore from the
backup, transactions are  replayed, similar to crash-recovery to bring the database
to a consistent state. New online backup also will work same way,  except
that all the transaction log must be copied to the backup, only after all the data 
files are backed up.


I think current implementation freezes(no changes to the database) the database 
during backup for following reasons :
1) Data files will in a stable state; backup will not contain partially updates
   pages on the disk. 
2) No new data files will be added/deleted on the disk; 
   because create/drop operations are blocked.
3) No transaction will committed after the backup starts. So all 
   unlogged operations will be rolled back. 

If the database is not frozen above conditions will not be true, that might
lead to the backups that are in corrupted/inconsistent state. I think, it is
not necessary to freeze the whole database to make a stable backup copy, by
blocking operations that modifies the files on-disk for small amounts of time, 
a stable backup can be made. 


Following sections explain some of the issues and possible ways to address them to
provide a real online backup that does not block changes to the database for the whole
duration of the backup.   

1) Corrupt pages in the backup database:

Backup reads and the  page cache writes can be interleaved if the database is
not frozen. i.e it is possible to land up with a page in the backup that  has  
a portion of the page that is more up-to-date than the rest of the page, if the
page cache writes are not blocked when a page is being read for the backup.


To avoid backup process reading partial written pages, some kind of
synchronization mechanism that  does not allow reading a page to write to the
back backup when the same page is being written to the disk.  This can be
implemented  by one of the following approaches:

a) By latching on a page key (container id, page number)  while doing the write 
   of the page from cache to disk  and while reading the page from the
   disk/cache to write to the backup. This approach has small overhead of
   acquiring an extra latch during the page cache writes when the backup is in progress. 

or 

b) read each pages in to the page cache first and then  latch  the
   page in the cache until a  temporary copy of it is made. This approach
   does not have extra overhead of extra latches on the page keys during writes , but 
   will pollute the page cache with the pages that are only required by the 
   backup; this  might have  impact on user operations because active user pages may 
   have been replaced by the backup pages in the page cache. 

or 

c) read pages into buffer pool and latch them while making a copy  similar to
the above approach,  but some how make sure that user pages are not kicked out 
of the buffer pool.   


One optimization  that may be made is to copy the file on the disk as it 
is to the backup, but keep track of pages that gets modified when file was 
being copied and rewrite those pages by using one of the above latching
mechanisms. 
 

2) Committed Non logged operation:
   
  Basic requirement to have consistent database backup is after the checkpoint
  for the backup all changes to the database will be available in the
  transaction log.  But Derby provides some non logged operations for
  performance reasons , for example  CREATE INDEX , IMPORT to a empty table
  ..etc. 
   
  This was not a issue in the old backup mechanism because no operations will
  be committed once the backup starts. So any non logged operations will be rolled 
  back similar to the regular crash recovery. 


  I can think of two ways to address this issue:

  a) To block non-logged operations when backup is in progress and also make backup
     wait before copying until the non-logged operation are complete. 
 
  b) make backup always wait for the non-logged operations to complete and
     retake the backup of those files that got affected by the non-logged
     operation, if they were already backed up.  
  
  c)  Some how trigger logging for all the operations after the checkpoint for
      the backup until the backup is  complete. This one is easy to implement 
      for non-logged operation  that are stated after the  backup, but the
      tricky case is to trigger logging for those non-logging operation that
      started before the backup but are committed during the backup. 

   
3) drop of a table when the  file on the disk  is being backed up. drop of 
   a table will result in deletion of the file on the disk, but deletion will get errors
   if it is opened for backup.  

   Some form of synchronization required to make sure that users do not see 
   weird errors in this case.  

4) creating a table/index  after the data files are backed up. Basically 
   recovery  system expects that file on the disk exists before the log records
   that  refer to it are written to the transaction log. 
   
   I think roll-forward recovery already handles this case , but should be
   tested. 

5) data file growth because of inserts when the file(table/index) is being backed up. 

   Recovery system  expects that a page is allocated on the disk 
   before log records are written to the transaction log about a page  to 
   avoid recovery errors because of space issues except incase of roll-forward
   recovery. 

   I think roll-forward recovery handles this case already; but have to make
   sure it will  work in this case also. Test cases should be added.

   Some form of synchronization is required, to make  a stable table snap shot of the
   file , if the file is growing when the backup is in progress.  

6) checkpoints when the backup is in progress.

   I think it not necessary to allow checkpoints when the backup is in
   progress. But if some one thinks otherwise , following should
   be addressed: 

   1) make copy of the log control file for the backup before copying any   
   2) If there are any operations that rely on checkpoint to make the
   operation consistent should not be allowed because backup might have 
   already copied some files when checkpoint happens.  


Any comments/suggestions will be appreciated. 


Thanks
-suresh

> Need a online backup feature  that does not block update operations   when online backup is in progress.
> --------------------------------------------------------------------------------------------------------
>
>          Key: DERBY-239
>          URL: http://issues.apache.org/jira/browse/DERBY-239
>      Project: Derby
>         Type: New Feature
>   Components: Store
>     Versions: 10.1.1.0
>     Reporter: Suresh Thalamati
>     Assignee: Suresh Thalamati

>
> Currently Derby allows users to perfoms  online backups using SYSCS_UTIL.SYSCS_BACKUP_DATABASE() procedure,  but while the backup is in progress, update operations are temporarily blocked, but read operations can still proceed.
> Blocking update operations can be real issue specifically in client server environments, because user requests will be blocked for a long time if a 
> backup is in the progress on the server.

-- 
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
   http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see:
   http://www.atlassian.com/software/jira